109Cafe

Move from Hosted Chef to Self Hosted Chef Infra Server

SysAdmOct 3, 2024

As Progress announced, Hosted Chef (aka manage.chef.io) will reach its EOL on November 30, 2024.

I assume you've compared Chef SaaS, Chef Automate, and Chef Infra Server. It seems you weren't happy with the high costs of Chef SaaS or Chef Automate, so you decided to host the open-source Chef Infra Server yourself.

Here's a guidance to deploy the open-source Chef Infra Server and migrate your Chef data from Hosted Chef to your self-hosted Chef Infra Server in just a few minutes.

Deployment

You'll deploy the Chef Infra Server using the chef-automate CLI.

This doesn't mean you need to deploy Chef Automate. The chef-automate CLI is only well-maintained method from Progress and the easiest, painlessly way to set up a standalone Chef Infra Server.

Here’s what you need to do:

1. Download the chef-automate cli:

server ~
curl https://packages.chef.io/files/current/latest/chef-automate-cli/chef-automate_linux_amd64.zip | gunzip - > chef-automate && chmod +x chef-automate

2. Generate a config file for deployment.

./chef-automate init-config

This command creates a file named config.toml, which is quite simple and looks like this:

config.toml
[global.v1]
  fqdn = "<redacted>"

  [[global.v1.frontend_tls]]
    cert = """<redacted>"""
    key = """<redacted>"""

[deployment.v1]
  [deployment.v1.svc]
    channel = "current"
    upgrade_strategy = "at-once"
    deployment_type = "local"

[license_control.v1]
  [license_control.v1.svc]
    license = ""

[opensearch.v1.sys.runtime]
  heapsize = "2g"

3. Update the configuration file to deploy only the Chef Infra Server:

[deployment.v1.svc]
products=["infra-server"]

# Disable Automate data collection as Automate will not be deployed
[erchef.v1.sys.data_collector]
enabled = false

Your config.toml should now look like this:

config.toml
[global.v1]
  fqdn = "<redacted>"

  [[global.v1.frontend_tls]]
    cert = """<redacted>"""
    key = """<redacted>"""

[deployment.v1]
  [deployment.v1.svc]
    channel = "current"
    upgrade_strategy = "at-once"
    deployment_type = "local"
    products=["infra-server"]

[license_control.v1]
  [license_control.v1.svc]
    license = ""

[opensearch.v1.sys.runtime]
  heapsize = "2g"

# Disable Automate data collection as Automate will not be deployed
[erchef.v1.sys.data_collector]
enabled = false

4. Deploy the Chef Infra Server

./chef-automate deploy config.toml

Wait a few minutes, and your Chef Infra Server will be ready.

In case you want to change the config after you have finished the deployment, after modify the config file, you have to run chef-automate config set or chef-automate config patch to apply the new config.

5. Check the status of the deployment

After finishing, the chef-automate and chef-server-ctl CLIs will be available in /usr/bin.

To check the status of each compoent:

chef-automate status
Status from deployment with channel [current] and type [local]

Service Name            Process State  Health Check  Uptime (s) PID
deployment-service      running        ok            234916     3913698
backup-gateway          running        ok            219970     4041343
automate-postgresql     running        ok            219970     4041393
automate-pg-gateway     running        ok            219968     4041584
automate-opensearch     running        ok            219968     4041606
automate-es-gateway     running        ok            219967     4041705
pg-sidecar-service      running        ok            219968     4041677
cereal-service          running        ok            219967     4041751
es-sidecar-service      running        ok            219913     4043345
license-control-service running        ok            219966     4041806
automate-cs-bookshelf   running        ok            219965     4041875
automate-cs-oc-bifrost  running        ok            219964     4041905
automate-cs-oc-erchef   running        ok            219911     4043385
automate-cs-ocid        running        ok            219964     4041942
automate-cs-nginx       running        ok            219963     4042103
automate-load-balancer  running        ok            219963     4042132

Visit https://<fqdn>/_status, and you should see:

{
  "status": "pong",
  "upstreams": {
    "chef_opensearch": "pong",
    "chef_sql": "pong",
    "oc_chef_authz": "pong"
  },
  "keygen": {
    "keys": 1000,
    "max": 1000,
    "max_workers": 1000,
    "cur_max_workers": 1000,
    "inflight": 0,
    "avail_workers": 1000,
    "start_size": 2
  },
  "indexing": {
    "mode": "batch"
  }
}

Migration

Now you're ready to migrate data from Hosted Chef to your self-hosted Chef Infra Server.

First, create an user and an organization using

Create user
chef-server-ctl user-create USERNAME FIRST_NAME [MIDDLE_NAME] LAST_NAME EMAIL PASSWORD --filename new_key.pem

Create organization
chef-server-ctl org-create ORG_SHORT_NAME ORG_FULL_NAME --filename new_org_validator_key.pem

It's better to keep the username and short organization name as before for a easier migration.

Then, create a new knife.rb for the new user created just now,similar to the one you used before.

You should now have an old config directory (A) with the knife.rb and PEM file for Hosted Chef, and a new config directory (B) with the knife.rb and PEM file for your self-hosted Chef Infra Server.

Run the following command to back up data from the Hosted Chef using config A to a directory named backup:

knife download --config A/knife.rb --chef-repo-path backup /

Next, run this command to restore data from the backup directory to your self-hosted Chef Infra Server using config B:

knife upload / --chef-repo-path backup --config B/knife.rb

Verify and Transition

To confirm the new setup, log into a server managed by Chef before, edit /etc/chef/client.rb for the new setup

1. update the chef_server_url to use the new Chef Infra Server

2. add data_collector.mode :solo to disable data collector

3. run chef-client to verify

Once the new setup is confirmed, the chef_client_config resource will help complete the transition for all servers. Here's an example:

default.rb
ssl_verify_mode = (node['chef_client']['ssl_verify_mode'] or ':verify_peer')
chef_client_config 'client' do
  chef_server_url node['chef_client']['chef_server_url']
  chef_license 'accept'
  file_cache_path '/var/chef/cache'
  file_backup_path '/var/chef/backup'
  ssl_verify_mode ssl_verify_mode
  additional_config "data_collector.mode	:solo
"
  log_location 'STDOUT'
end

Further Information

1. Why chef-automate CLI

There're ~3 methods that you will find to setup a standalone Chef Infra Server:

a. The main document of Chef Infra Server guides you to install version 12.x, which is quite outdated.

b. Follow the main GitHub repository of Chef Infra Server will waste your time on bootstrapping components of Chef Server with combined latest and deprecated habitat builds, with little guidance and documentation.

c. So the sub-document of Chef Automate wins in the end.

2. Requirements of a standalone Chef Infra Server

It varies.

I run Chef Infra Server on a cloud server with 2 shared vCPUs and 4GB of ram, setting the [opensearch.v1.sys.runtime].heapsize to 1g, to manage 10 nodes.

By using an external PostgreSQL database and opensearch/elasticsearch instance, the requirements can be much lower.

Referral: Try cloud computing services for free with a $100 credit

3. FDQN, Hostname and SSL Problem

I didn't say much about FQDN, hostname, and the SSL issue in this article since I assume you have experience with server management.

Once you have set up the domain and received the SSL certificate, just place the certificate and key in the [global.v1.frontend_tls] section.