kitchen-google

A Test Kitchen driver that creates and destroys Google Compute Engine instances, so you can test your cookbooks and infrastructure code on real GCE VMs.

Compared to other IaaS providers, GCE offers fast instance launch times and sub-hour billing, which makes it well suited to short-lived test instances.

This documentation uses Cinc Workstation and the cinc commands throughout. Everything here works identically with Chef Workstation — see Using with Chef.

Contents

• Requirements
• Installation
• Authentication
• Quick Start
• Configuration
• Examples
• Troubleshooting
• Using with Chef
• Contributing
• License

Requirements

• Ruby 3.1 or later (already satisfied if you use Cinc Workstation)
• Test Kitchen 3.0 or later
• A Google Cloud project with the Compute Engine API enabled
• Credentials available to Google Application Default Credentials

Installation

This driver ships as part of Cinc Workstation. If you have Cinc Workstation installed, there is nothing else to install.

To install it into a standalone Ruby:

gem install kitchen-google

Or with Bundler, add it to your Gemfile:

gem "kitchen-google"

...then run bundle install.

Provisioners and verifiers

Test Kitchen 4.0 stopped bundling the Chef provisioners. It now ships only the dummy, external and shell provisioners, and the busser, dummy and shell verifiers — so the plugins named in the examples below come from separate gems, whatever driver you use.

If you use Cinc Workstation or Chef Workstation, they are already installed and there is nothing to do.

In a standalone Ruby, install the ones your kitchen.yml names alongside this driver:

kitchen.yml value	Gem
provisioner: cinc_infra	kitchen-cinc
provisioner: chef_infra	kitchen-cinc (runs Cinc Client) or kitchen-omnibus-chef (runs Chef Infra Client)
verifier: inspec	kitchen-inspec
verifier: cinc_auditor	ships with Cinc Workstation

gem "kitchen-google"
gem "kitchen-cinc"    # provisioner
gem "kitchen-inspec"  # verifier

Omitting them fails in Test Kitchen before this driver is ever reached:

Could not load the 'cinc_infra' provisioner from the load path. Did you mean:
dummy, external, shell ? Please ensure that your provisioner is installed as a
gem or included in your Gemfile if using Bundler.

On Test Kitchen 3.x the chef_* provisioners were built in, so this is a common surprise when upgrading.

Authentication

The driver authenticates using Application Default Credentials. Set them up once with the gcloud CLI:

gcloud auth application-default login

Alternatively, point GOOGLE_APPLICATION_CREDENTIALS at a service account key file:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json

The account needs permission to create and delete instances and disks in the target project.

Quick Start

Create a kitchen.yml in your cookbook:

---
driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud
  machine_type: n1-standard-1
  tags:
    - test-kitchen

provisioner:
  name: cinc_infra

verifier:
  name: cinc_auditor

platforms:
  - name: ubuntu-22.04

suites:
  - name: default
    run_list:
      - recipe[my_cookbook::default]

Then run the full test cycle:

cinc kitchen test

Or step through it:

cinc kitchen create    # launch the GCE instance
cinc kitchen converge  # apply your cookbook
cinc kitchen verify    # run your tests
cinc kitchen destroy   # delete the instance

Checking whether an instance is still there

kitchen list reports what the last action did, which is not the same as what GCE currently holds. On Test Kitchen 4 or later, --live asks GCE directly:

cinc kitchen list --live

Instance              Driver                ...  Last Action  Live Status
baseline-ubuntu-2204  Google Compute (GCE)  ...  Created      terminated

The reported state is GCE's own — running, terminated, provisioning, suspended and so on — plus not created when nothing has been launched and not found when the state file names an instance that no longer exists.

This is worth knowing about if you use preemptible: true, since GCE can reclaim the instance at any time and Test Kitchen's own record will still say Created. kitchen list --live --json includes the instance name and the time of the check.

Configuration

All options below are set under the driver: key in kitchen.yml.

Required

Option	Default	Description
project	none	GCP project ID in which to create instances. Required.

You must also specify either zone or region, and either image_family or image_name.

Location

Option	Default	Description
zone	nil	Zone to launch the instance in, e.g. us-central1-a. Takes precedence over region.
region	nil	Region to launch in, e.g. us-central1. A zone within the region is chosen at random. Ignored if zone is set. The value any is no longer supported.

Image

Option	Default	Description
image_family	nil	Image family to boot from; the latest non-deprecated image in the family is used, e.g. ubuntu-2204-lts. Ignored if image_name is set.
image_name	nil	Exact image name to boot from. Takes precedence over image_family.
image_project	nil	Project that owns the image. If unset, only the current project is searched.

Machine

Option	Default	Description
machine_type	"n1-standard-1"	GCE machine type, e.g. n1-standard-1, e2-medium.
inst_name	nil	Explicit instance name. If unset, a unique name is generated from the suite and platform.
preemptible	false	Launch as a preemptible instance.
auto_restart	false	Automatically restart the instance if it is terminated by GCE. Forced off for preemptible instances.
auto_migrate	true	Live-migrate the instance during host maintenance. Forced off for preemptible instances and for instances with guest_accelerators, neither of which GCE will migrate. Some machine families, including e2-*, reject false on a non-preemptible instance.
guest_accelerators	[]	Array of accelerator (GPU) hashes, each with type and count keys.
metadata	{}	Instance metadata key/value pairs. Merged over the metadata the driver sets itself (created-by, test-kitchen-instance, test-kitchen-user, and windows-startup-script-ps1 on Windows), so setting one of those keys replaces the driver's value and logs a warning.
labels	{}	Labels applied to the instance as key/value pairs.
tags	[]	Network tags applied to the instance, used by firewall rules.

Network

Option	Default	Description
network	"default"	Network to attach the instance to.
network_project	nil	Project that owns the network. Defaults to project. Used with Shared VPC.
network_ip	nil	Specific internal IP address to assign.
subnet	nil	Subnetwork to attach to, required for custom-mode VPC networks.
subnet_project	nil	Project that owns the subnet. If unset, only the current project is searched.
use_private_ip	false	Connect to the instance over its internal IP and do not assign an external IP.

Service account

Option	Default	Description
service_account_name	"default"	Service account attached to the instance.
service_account_scopes	[]	Array of OAuth scopes granted to the instance. Accepts short aliases such as storage-ro, compute-rw, cloud-platform, logging-write, or full scope URLs.
email	nil	Email address of the GCE user. Required when using the WinRM transport, for Windows password generation.

Disks

Disks are configured with the disks hash. Each key is a disk name, and each value accepts the options below.

Option	Default	Description
disks	one 10 GB boot disk	Hash of disks to attach, keyed by disk name. Disk names must match [a-z]([-a-z0-9]*[a-z0-9])?.
disks.<name>.boot	first eligible disk	Marks this disk as the boot disk. At most one disk may set boot: true. If none does, the first eligible disk is used — skipping local SSDs, which cannot boot, and any disk that sets boot: false.
disks.<name>.disk_size	10	Size in GB. Raised automatically when the image it is created from is larger. Must be omitted for local-ssd, which is always 375 GB.
disks.<name>.disk_type	chosen by GCE	Disk type, e.g. pd-balanced, pd-ssd, hyperdisk-balanced, local-ssd. See disk types and machine series.
disks.<name>.autodelete_disk	true	Delete the disk when the instance is destroyed.
disks.<name>.custom_image	nil	Image to create a non-boot disk from. Looked up in image_project, the same project as the boot image, so if you boot from a public image such as ubuntu-os-cloud your own images are not visible here. Creation fails if the image cannot be found.

Example:

driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud
  disks:
    boot-disk:
      boot: true
      disk_size: 20
      disk_type: pd-ssd
    data-disk:
      disk_size: 100

Disk types and machine series

disk_type is left unset unless you configure it, and GCE then picks the default for the instance's machine series: pd-standard on first- and second-generation series such as N1, N2 and E2, pd-balanced on C3, C3D and M3, and hyperdisk-balanced on C4, N4 and newer. This is why no disk_type is set by default — newer machine series reject the older disk types outright, so any fixed default would break some machine type.

Set disk_type explicitly only when you want something other than that default, and check that the type you choose is supported by your machine_type.

One exception is worth knowing about. Extra non-boot disks are created as standalone disks before the instance exists, so GCE has no machine series to derive a default from and falls back to pd-standard regardless of the instance's machine_type. On Hyperdisk-only machine series such as C4 and N4, set disk_type explicitly on those disks:

driver:
  name: gce
  machine_type: n4-standard-2
  disks:
    boot-disk:
      boot: true
    data-disk:
      disk_size: 100
      disk_type: hyperdisk-balanced

Deprecated top-level disk options

The options below configure a single boot disk and are kept for backwards compatibility. They cannot be combined with disks.

Option	Default	Description
disk_size	10	Deprecated. Boot disk size in GB, raised automatically when the image is larger. Use disks instead.
disk_type	chosen by GCE	Deprecated. Boot disk type. Use disks instead.
autodelete_disk	true	Deprecated. Delete the boot disk on destroy. Use disks instead.

Timing

Option	Default	Description
wait_time	600	Seconds to wait for a GCE operation to finish or a resource to reach a status - creating and deleting the instance, and creating disks.
refresh_rate	2	Seconds between status checks while waiting.
winpass_timeout	120	Seconds to wait for the Windows guest agent to reset the password.

wait_time does not cover waiting for the instance to accept connections. That wait belongs to the transport, which applies its own max_wait_until_ready (600 seconds by default) on top of its per-attempt connection_timeout, so an instance that is unreachable rather than merely slow can be waited on for considerably longer than wait_time:

transport:
  name: ssh
  max_wait_until_ready: 120

This is worth setting if you use use_private_ip from outside the network, or WinRM without a firewall rule for port 5985 - in both cases the transport can never connect, and the default is a long time to spend finding that out on a running instance.

Examples

Multiple platforms

driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  image_project: ubuntu-os-cloud

platforms:
  - name: ubuntu-22.04
    driver:
      image_family: ubuntu-2204-lts
  - name: ubuntu-24.04
    driver:
      image_family: ubuntu-2404-lts
  - name: centos-stream-9
    driver:
      image_family: centos-stream-9
      image_project: centos-cloud

Preemptible instances to reduce cost

driver:
  name: gce
  project: my-gcp-project
  region: us-central1
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud
  preemptible: true

Private IP only, on a Shared VPC

driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud
  network: shared-net
  network_project: my-host-project
  subnet: shared-subnet
  subnet_project: my-host-project
  use_private_ip: true

Windows

Windows guests need three things beyond the Linux setup: an email address, a username that is not Administrator, and a firewall rule that lets WinRM in.

driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  image_family: windows-2022
  image_project: windows-cloud
  email: me@example.com
  tags:
    - test-kitchen-winrm

transport:
  name: winrm
  # Not "Administrator" - see below.
  username: kitchen

platforms:
  - name: windows-2022

email identifies the requesting user in the key exchange the driver uses to reset the password. It is required, and validation fails without it.

username must not be the built-in Administrator account, which is Test Kitchen's default for the WinRM transport. Google's Windows images ship that account disabled, and the guest agent resets its password without enabling it, so the login is refused and the run dies at WinRM::WinRMAuthorizationError. Any other name works: the agent creates the account, adds it to the local Administrators group, and enables it. The driver warns if it sees this before you have to wait out the failure.

Firewall. Nothing in the default VPC allows WinRM. The default-allow-* rules on an auto mode network cover SSH, RDP and ICMP, but not TCP 5985, so without a rule of your own kitchen create hangs at "Waiting for server to be ready" until it times out. The driver adds a matching rule inside the guest via a startup script, but that cannot open the VPC. Create the rule once per project, and tag the instances with tags so it applies to them:

gcloud compute firewall-rules create test-kitchen-winrm \
  --project my-gcp-project \
  --allow tcp:5985 \
  --target-tags test-kitchen-winrm \
  --source-ranges 0.0.0.0/0

Narrow --source-ranges to the addresses you run Test Kitchen from rather than leaving it open to the internet.

Attaching a GPU

driver:
  name: gce
  project: my-gcp-project
  zone: us-central1-a
  machine_type: n1-standard-4
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud
  guest_accelerators:
    - type: nvidia-tesla-t4
      count: 1

Troubleshooting

Almost every failure below is reported by the driver before it creates anything, so nothing is billing while you work out what is wrong.

Project my-project is not a valid project. Usually credentials rather than the project name: the driver cannot see a project it is not authorised for, and reports that the same way as one that does not exist. Check with gcloud auth application-default print-access-token, then confirm the Compute Engine API is enabled on the project:

gcloud services enable compute.googleapis.com --project my-gcp-project

Either zone or region must be specified. Neither has a default. Set zone for a specific zone, or region to let the driver pick a zone that is up. region: any was removed.

Either image family or name must be specified. Same again — set image_family for the current image in a family, or image_name to pin one exactly.

Image family ubuntu-2204-lts is not valid - it was not found in project my-gcp-project. The family exists, but not in the project the driver searched. Public images live in their own projects, so image_project is almost always needed alongside image_family:

driver:
  image_family: ubuntu-2204-lts
  image_project: ubuntu-os-cloud   # not your own project

gcloud compute images list shows which project owns each family. The same applies to custom_image on a non-boot disk, which is resolved in image_project too — so if you boot from a public image, your own images are not visible to it.

Machine type e2-medium is not valid. Machine types are per-zone, and not every family exists in every zone. gcloud compute machine-types list --zones us-central1-a lists the ones you can actually use.

Disk type pd-standard for disk boot is not valid, or a create that fails on diskType. Disk types are per-zone as well, and newer machine series reject the older ones outright. Leave disk_type unset unless you need something specific — see Disk types and machine series.

Instance name ... is not valid. GCE names must start with a lowercase letter and end with a lowercase letter or a digit. The driver folds case and substitutes anything else, but it cannot fix a name that starts with a digit or an underscore. This comes from inst_name, or from a suite or platform name that begins with something GCE will not accept.

Disk name ... is too long. Disk names are <instance name>-<disk name> and share the same 63-character budget, so a long key under disks: leaves no room for the instance name. Shorten the disk name.

Unable to find a suitable zone in us-central1. No zone in the region reported itself as UP. Check the region name, and Google Cloud status.

Quota errors on create. Quota 'CPUS' exceeded and friends come back from GCE, not from the driver. Quotas are per-region: check gcloud compute regions describe us-central1, and remember that a failed run that was interrupted may still be holding instances.

kitchen create hangs at "Waiting for server to be ready". The instance is running and the transport cannot reach it. In order of likelihood:

• A firewall rule. The default VPC allows SSH and RDP but not WinRM. Add a rule for TCP 5985 and tag the instances — see Windows.
• use_private_ip: true from outside the network. There is no route to the internal address unless you are on the VPC or behind a VPN.
• OS Login. If the project or the instance enforces it, keys published as ssh-keys metadata are ignored. Either grant the account roles/compute.osLogin and let gcloud compute ssh provision it, or set enable-oslogin: "FALSE" in the instance metadata.

The wait itself belongs to the transport, not to wait_time — see Timing. Lowering max_wait_until_ready turns a ten-minute hang into a two-minute failure while you work out which of the three it is.

WinRM::WinRMAuthorizationError on a Windows platform. The transport is connecting as Administrator, which Google's images ship disabled. The guest agent resets its password without enabling it, so the login is refused. Set transport.username to anything else and the agent creates that account instead. The driver warns about this before it creates the instance.

Timed out after 120 seconds waiting for the GCE agent to reset the password. The in-guest agent never answered on the serial port. Windows images take several minutes to first boot, so raise winpass_timeout. If it still times out, the image probably has no guest agent — use one from windows-cloud rather than a custom image built without it.

Request did not complete in 600 seconds. A GCE operation did not finish within wait_time. The operation is still running on Google's side, so look at the instance in the Cloud Console before retrying.

Instances or disks left behind after an interrupted run. Ctrl-C raises Interrupt, which is not a StandardError, so the driver's own cleanup does not run. It records what it created in the state file first, precisely so that a follow-up kitchen destroy can find it:

cinc kitchen destroy

If the state file is gone too, the driver stamps created-by: test-kitchen into every instance's metadata, so they can be found and removed by hand:

gcloud compute instances list --filter="metadata.items.key=created-by AND metadata.items.value=test-kitchen"
gcloud compute disks list --filter="-users:*"

Anything else. Run with -l debug:

cinc kitchen create default-ubuntu-2204 -l debug

The debug log records every API call the driver makes and the error the Google client returned, which is usually enough to see what GCE actually objected to.

Using with Chef

This driver is not tied to Cinc. The examples above use Cinc Workstation and the cinc_infra provisioner, but the driver works exactly the same with Chef Workstation — run kitchen instead of cinc kitchen, and use chef_infra instead of cinc_infra:

provisioner:
  name: chef_infra

verifier:
  name: inspec

No driver configuration changes are needed.

Contributing

Bug reports and pull requests are welcome on GitHub. See CONTRIBUTING.md for development setup, how to run the tests, and the release process.

Acknowledgements

Created and maintained by Andrew Leonard (andy@hurricane-ridge.com). The initial release drew heavily on the kitchen-ec2 gem for both inspiration and implementation details.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.