From b07c8afffd61c58938066e3afb208e10e0f7c6c2 Mon Sep 17 00:00:00 2001 From: Diego OJ Date: Sun, 30 Aug 2026 17:59:18 -0300 Subject: [PATCH 1/2] Fix | Restore the integration tests Fix `leverage project create`: The root command builds the project paths for every invocation, which requires an already configured project. `project init` creates the git repository, so from that point on the configuration loads fine but still holds no project name, and `project create` aborts with "Project name has not been set" on what is its normal, intended use. The command that creates a project could not run on a project that did not exist yet. Before #316 the root command only loaded the config to check the toolbox version, and returned early when it was absent, with a comment noting that the config does not exist yet at some points of the project. That guard was lost when the paths were introduced. The project commands now skip the paths setup, which they never use. A unit test covers the regression. Run the bats tests on the runner, and drop the testing image: The image existed to run the cli inside a container. Since #316 the cli runs the binaries on the host, and the image was never given terraform or tofu, so `leverage terraform version` had been failing on every run since December 2025. What was left of the image was a Linux box with python, bats and leverage in it, which is what the runner already is, and which the unit test workflow already relies on. Dropping it also removes a dind base that no longer had a purpose, the `--privileged` flag and the dockerd entrypoint it needed, and a bind mount that had no effect since the image ran out of the copy made at build time. The no-git test used to `apk del git`, tying it to the image and to the file ordering of the suite. It now runs the cli with a PATH holding nothing but its own entry point, which `shutil.which` resolves the same way, without touching the system. Two portability fixes let the suite run on macOS as well: an explicit mktemp template, since `-t` yields a name with a dot that breaks the module name the build script is imported under, and POSIX classes instead of `\s`, which only glibc accepts. Drop TERRAFORM_IMAGE_TAG from the bats fixtures, unread since #316. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/tests-integration.yaml | 39 +++++++++++++++++++++--- CLAUDE.md | 10 ++---- Dockerfile | 32 ------------------- Makefile | 14 +++------ README.md | 22 ++++++------- entrypoint.sh | 7 ----- leverage/leverage.py | 7 +++++ tests/bats/leverage.bats | 10 +++--- tests/bats/leverage_terraform.bats | 21 ++++++++++--- tests/bats/no_git_leverage.bats | 24 ++++++--------- tests/bats/utils.bash | 7 +++-- tests/test_modules/test_project.py | 32 +++++++++++++++++++ 12 files changed, 127 insertions(+), 98 deletions(-) delete mode 100644 Dockerfile delete mode 100755 entrypoint.sh diff --git a/.github/workflows/tests-integration.yaml b/.github/workflows/tests-integration.yaml index 117c82c4..5fafd251 100644 --- a/.github/workflows/tests-integration.yaml +++ b/.github/workflows/tests-integration.yaml @@ -5,14 +5,43 @@ on: [pull_request, workflow_dispatch] jobs: integration_tests: runs-on: ubuntu-latest + env: + TERRAFORM_VERSION: 1.9.8 + OPENTOFU_VERSION: 1.9.1 + BATS_LIB_PATH: /usr/local/lib/bats steps: - - uses: actions/checkout@v3 + - uses: actions/checkout@v4 - - name: build_image + - uses: actions/setup-python@v5 + with: + python-version: '3.13' + + - name: Install Leverage CLI run: | - echo "[INFO] Building image" - make build-image - shell: bash + echo "[INFO] Installing the cli, the bats tests drive the installed command" + pip install -e . + leverage --version + + - name: Install bats + run: | + echo "[INFO] Installing bats and its libraries" + git clone --depth 1 https://github.com/bats-core/bats-core.git /tmp/bats-core + sudo /tmp/bats-core/install.sh /usr/local + sudo mkdir -p "${BATS_LIB_PATH}" + sudo git clone --depth 1 https://github.com/bats-core/bats-support.git "${BATS_LIB_PATH}/bats-support" + sudo git clone --depth 1 https://github.com/bats-core/bats-assert.git "${BATS_LIB_PATH}/bats-assert" + + - name: Install terraform and tofu + run: | + echo "[INFO] Installing terraform ${TERRAFORM_VERSION} and tofu ${OPENTOFU_VERSION}" + curl -fsSL -o /tmp/terraform.zip \ + "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" + curl -fsSL -o /tmp/tofu.zip \ + "https://github.com/opentofu/opentofu/releases/download/v${OPENTOFU_VERSION}/tofu_${OPENTOFU_VERSION}_linux_amd64.zip" + sudo unzip -q -o /tmp/terraform.zip terraform -d /usr/local/bin + sudo unzip -q -o /tmp/tofu.zip tofu -d /usr/local/bin + terraform version + tofu version - name: run_integration_tests run: | diff --git a/CLAUDE.md b/CLAUDE.md index 342e559d..d2aae085 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,9 +11,9 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ### Testing - `poetry run pytest` - Run unit tests - `poetry run pytest --verbose --cov=./ --cov-report=xml` - Run unit tests with coverage -- `make test-unit` - Run unit tests in Docker (with coverage) -- `make test-unit-no-cov` - Run unit tests in Docker (no coverage) -- `make test-int` - Run integration tests using bats in Docker +- `make test-unit` - Run unit tests (with coverage) +- `make test-unit-no-cov` - Run unit tests (no coverage) +- `make test-int` - Run integration tests using bats (requires bats, terraform and tofu) - `make tests` - Run full test suite (unit + integration) ### Code Quality @@ -28,10 +28,6 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co - `poetry build` - Build package using Poetry - `make clean` - Clean build artifacts -### Docker -- `make build-image` - Build Docker testing image -- All test commands can run in Docker using the testing image - ## Architecture Leverage CLI is a Python-based command-line tool for managing Binbash Leverage projects. It uses host-based execution to run infrastructure tools directly on the system. diff --git a/Dockerfile b/Dockerfile deleted file mode 100644 index bef7e5b6..00000000 --- a/Dockerfile +++ /dev/null @@ -1,32 +0,0 @@ -FROM docker:24.0.7-dind-alpine3.18 - -LABEL vendor="Binbash Leverage (leverage@binbash.com.ar)" - -RUN apk update &&\ - apk add --no-cache bash bash-completion ncurses git curl gcc musl-dev python3 python3-dev py3-pip - -ENV POETRY_VIRTUALENVS_CREATE=false -ENV PATH="${PATH}:/root/.poetry/bin" - -# Install bats from source -RUN git clone https://github.com/bats-core/bats-core.git && ./bats-core/install.sh /usr/local -# Install other bats modules -RUN git clone https://github.com/bats-core/bats-support.git -RUN git clone https://github.com/bats-core/bats-assert.git - -# Needed as is mounted later on -RUN mkdir /root/.ssh - -RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/usr/local POETRY_VERSION=1.8.2 python3 - - -RUN git config --global --add safe.directory /workdir - -# Copying all necessary files to /workdir directory -COPY . /workdir -WORKDIR /workdir - -RUN poetry install --with=dev --with=main - -COPY entrypoint.sh / -# Make script to configure and start docker daemon the default entrypoint -ENTRYPOINT [ "/entrypoint.sh" ] diff --git a/Makefile b/Makefile index 2d5aa264..ee773f43 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,4 @@ .PHONY: help build -LEVERAGE_TESTING_IMAGE := binbash/leverage-cli-testing -LEVERAGE_TESTING_TAG := 2.5.0 -LEVERAGE_IMAGE_TAG := 1.3.5-0.2.0 PYPROJECT_FILE := pyproject.toml INIT_FILE := leverage/__init__.py PLACEHOLDER := 0.0.0 @@ -25,17 +22,14 @@ help: @echo 'Available Commands:' @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | $(SORT) | awk 'BEGIN {FS = ":.*?## "}; {printf " - \033[36m%-18s\033[0m %s\n", $$1, $$2}' -build-image: ## Build docker image for testing - docker build . -t ${LEVERAGE_TESTING_IMAGE}:${LEVERAGE_TESTING_TAG} - test-unit: ## Run unit tests and create a coverage report - docker run --rm --privileged --mount type=bind,src=$(shell pwd),dst=/leverage -t ${LEVERAGE_TESTING_IMAGE}:${LEVERAGE_TESTING_TAG} pytest --verbose --cov=./ --cov-report=xml + pytest --verbose --cov=./leverage/ --cov-report=xml test-unit-no-cov: ## Run unit tests with no coverage report - docker run --rm --privileged --mount type=bind,src=$(shell pwd),dst=/leverage -t ${LEVERAGE_TESTING_IMAGE}:${LEVERAGE_TESTING_TAG} pytest --verbose --no-cov + pytest --verbose --no-cov -test-int: ## Run integration tests - docker run --rm --privileged --mount type=bind,src=$(shell pwd),dst=/leverage --env LEVERAGE_IMAGE_TAG=${LEVERAGE_IMAGE_TAG} -t ${LEVERAGE_TESTING_IMAGE}:${LEVERAGE_TESTING_TAG} bash -c "bats --verbose-run --show-output-of-passing-tests --print-output-on-failure -T -t -p -r tests/bats" +test-int: ## Run integration tests (requires bats, terraform and tofu, see README) + bats --verbose-run --show-output-of-passing-tests --print-output-on-failure -T -t -p -r tests/bats tests: test-unit-no-cov test-int ## Run full set of tests diff --git a/README.md b/README.md index 02265208..a571717b 100644 --- a/README.md +++ b/README.md @@ -183,12 +183,8 @@ poetry run pre-commit install To run unit tests, pytest is the tool of choice, and the required dependencies are available in the corresponding `dev-requirements.txt`. -Integration tests are implemented using [bats](https://github.com/bats-core/bats-core/). Bear in mind that bats tests -are meant to be run in a throwaway environment since they perform filesystem manipulations and installation and removal -of packages, and the cleanup may not be completely thorough. As such, is highly recommended to run these tests using the -docker image. - -### Manually +Integration tests are implemented using [bats](https://github.com/bats-core/bats-core/). They drive the installed +`leverage` command, and work on temporary directories of their own, so they can be run directly on your machine. 1. Unit tests: @@ -207,17 +203,21 @@ brew install bats-support brew install bats-assert ``` +The cli runs the infrastructure binaries directly, so `terraform` and `tofu` need to be installed as well. See +[System requirements](#system-requirements). + ```bash bats -r tests/bats ``` -### Using docker image +If `bats-support` and `bats-assert` are not installed in a location bats searches by default, point `BATS_LIB_PATH` at +the directory holding them: -A Docker image suitable for running all tests can be crafted by running `make build-image`. After crafting the image all -tests can be executed. +```bash +BATS_LIB_PATH=/opt/homebrew/lib bats -r tests/bats +``` -To run all tests, run `make tests`. Alternatively `make test-unit` or `make test-int` for unit or integration tests -respectively. +Alternatively, `make tests` runs both suites, and `make test-unit` or `make test-int` runs one of them. ## Release Process diff --git a/entrypoint.sh b/entrypoint.sh deleted file mode 100755 index c1069e30..00000000 --- a/entrypoint.sh +++ /dev/null @@ -1,7 +0,0 @@ -#!/bin/bash -# Configure docker daemon to listen through socket -mkdir /etc/docker -echo '{"tls": false, "hosts": ["unix:///var/run/docker.sock"]}' > /etc/docker/daemon.json -# Start daemon silently -dockerd > /dev/null 2>&1 & -exec "$@" diff --git a/leverage/leverage.py b/leverage/leverage.py index fd3da876..214b6b9a 100644 --- a/leverage/leverage.py +++ b/leverage/leverage.py @@ -27,6 +27,13 @@ def leverage(context, state, verbose): state.config = conf.load() except NotARepositoryError: return + + # The `project` commands bootstrap a project, so they run before its configuration exists. + # `project init` creates the git repository, so from that point on the config loads fine but + # still holds no project name, and building the paths would fail on a legitimate invocation. + if context.invoked_subcommand == project.name: + return + state.paths = PathsHandler(state.config) state.environment = { "AWS_SHARED_CREDENTIALS_FILE": str(state.paths.aws_credentials_file), diff --git a/tests/bats/leverage.bats b/tests/bats/leverage.bats index 7e027772..4f4a94f2 100644 --- a/tests/bats/leverage.bats +++ b/tests/bats/leverage.bats @@ -3,9 +3,9 @@ setup_file(){ } setup(){ - # Bats modules are installed globally - load "/bats-support/load.bash" - load "/bats-assert/load.bash" + # Resolved through BATS_LIB_PATH + bats_load_library bats-support + bats_load_library bats-assert # Store useful paths TESTS_ROOT="$( cd "$( dirname "$BATS_TEST_FILENAME" )/.." >/dev/null 2>&1 && pwd )" @@ -38,7 +38,7 @@ teardown(){ run leverage run -l assert_line --partial "Tasks in build file \`build.py\`:" - assert_line --regexp "hello\s+Say hello." + assert_line --regexp "hello[[:space:]]+Say hello." assert_line --regexp "Powered by Leverage [0-9]+.[0-9]+.[0-9]+" } @@ -52,7 +52,7 @@ teardown(){ run leverage run -l assert_line --partial "Tasks in build file \`build.py\`:" - assert_line --regexp "hello\s+Say hello." + assert_line --regexp "hello[[:space:]]+Say hello." assert_line --regexp "Powered by Leverage [0-9]+.[0-9]+.[0-9]+" } diff --git a/tests/bats/leverage_terraform.bats b/tests/bats/leverage_terraform.bats index 27a18b0f..f17666b7 100644 --- a/tests/bats/leverage_terraform.bats +++ b/tests/bats/leverage_terraform.bats @@ -1,7 +1,7 @@ setup(){ - # Bats modules are installed globally - load "/bats-support/load.bash" - load "/bats-assert/load.bash" + # Resolved through BATS_LIB_PATH + bats_load_library bats-support + bats_load_library bats-assert # Store useful paths TEST_ROOT="$( cd "$( dirname "$BATS_TEST_FILENAME" )/.." >/dev/null 2>&1 && pwd )" @@ -14,7 +14,7 @@ teardown(){ cd "$TESTS_ROOT" } -@test "Pulls terraform image and prints version" { +@test "Prints terraform version" { ROOT_DIR=$(_create_leverage_directory_structure) # Create required build.env in root directory and go there @@ -22,5 +22,16 @@ teardown(){ run leverage terraform version - assert_output --regexp "[\S\s]*Terraform v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2}[\s\S]*" + assert_output --regexp "Terraform v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2}" +} + +@test "Prints tofu version" { + ROOT_DIR=$(_create_leverage_directory_structure) + + # Create required build.env in root directory and go there + cd "$ROOT_DIR" + + run leverage tofu version + + assert_output --regexp "OpenTofu v[0-9]{1,2}\.[0-9]{1,2}\.[0-9]{1,2}" } \ No newline at end of file diff --git a/tests/bats/no_git_leverage.bats b/tests/bats/no_git_leverage.bats index 04ce7e51..d7fee09f 100644 --- a/tests/bats/no_git_leverage.bats +++ b/tests/bats/no_git_leverage.bats @@ -1,22 +1,18 @@ -# The name of the file was chosen as to avoid repetition of the leverage package installing step -# in leverage.bats, since bats respects file name order to run the tests - setup(){ - # Bats modules are installed globally - load "/bats-support/load.bash" - load "/bats-assert/load.bash" - - # Uninstall git - apk del git >/dev/null 2>&1 -} + # Resolved through BATS_LIB_PATH + bats_load_library bats-support + bats_load_library bats-assert -teardown(){ - # Reinstall git - apk add git >/dev/null 2>&1 + # A directory holding nothing but the leverage entry point, to be used as the whole PATH. + # The cli looks for git through `shutil.which`, which resolves it via PATH, and the console + # script has an absolute shebang, so its interpreter remains reachable. + GITLESS_PATH="$BATS_TEST_TMPDIR/gitless" + mkdir -p "$GITLESS_PATH" + ln -sf "$(command -v leverage)" "$GITLESS_PATH/leverage" } @test "Does not run if git is not installed in the system" { - run leverage + run env PATH="$GITLESS_PATH" leverage assert_failure assert_output "No git installation found in the system. Exiting." diff --git a/tests/bats/utils.bash b/tests/bats/utils.bash index 4499cec5..fa2631fd 100644 --- a/tests/bats/utils.bash +++ b/tests/bats/utils.bash @@ -7,8 +7,11 @@ _create_directory_structure(){ └ account And print the root path " - ROOT_DIR=$(mktemp -d -t tmpXXXXXX) - printf "PROJECT=ts\nTERRAFORM_IMAGE_TAG=%s\n" "$LEVERAGE_IMAGE_TAG" > $ROOT_DIR/"build.env" + # An explicit template keeps the name free of dots on both GNU and BSD mktemp. `-t` yields + # `tmp.XXXX` on macOS, and the build script is imported under a module name derived from the + # directory, which a dot turns into a package lookup that fails. + ROOT_DIR=$(mktemp -d "${TMPDIR:-/tmp}/leverageXXXXXX") + printf "PROJECT=ts\n" > $ROOT_DIR/"build.env" mkdir -p "$ROOT_DIR/config" mkdir -p "$ROOT_DIR/account" echo $ROOT_DIR diff --git a/tests/test_modules/test_project.py b/tests/test_modules/test_project.py index 6ba08d34..6fdf465e 100644 --- a/tests/test_modules/test_project.py +++ b/tests/test_modules/test_project.py @@ -1,9 +1,41 @@ +import subprocess +from pathlib import Path + import pytest +from click.testing import CliRunner +from leverage import conf, leverage +from leverage import path as lepath from leverage._utils import ExitError from leverage.modules.project import validate_config +def test_project_commands_run_before_the_project_configuration_exists(tmp_path, monkeypatch): + """ + Test that the project commands do not require an already configured project. + + `project init` creates the git repository, so from that point on the configuration loads fine + but holds no project name yet. Building the paths there aborts `project create` with + "Project name has not been set", on what is a perfectly legitimate invocation. + """ + root = tmp_path / "new-project" + root.mkdir() + subprocess.run(["git", "init"], cwd=root, check=True, capture_output=True) + + monkeypatch.setattr(lepath, "get_root_path", lambda: root) + monkeypatch.setattr(lepath, "get_working_path", lambda: root) + monkeypatch.setattr(conf, "get_root_path", lambda: root) + monkeypatch.setattr(conf, "get_working_path", lambda: root) + monkeypatch.setattr(Path, "cwd", lambda: root) + + result = CliRunner().invoke(leverage, ["project", "create"]) + + # The command is reached, and reports the missing configuration file on its own terms, rather + # than the run being aborted earlier while building the project paths. + assert "Project name has not been set" not in result.output + assert "No configuration file found for the project" in result.output + + @pytest.mark.parametrize( "project_name,short_name", [ From 264dcd88b9e85c8640bb948173117b528950178b Mon Sep 17 00:00:00 2001 From: Diego OJ Date: Sun, 30 Aug 2026 18:02:39 -0300 Subject: [PATCH 2/2] Fix | Do not require a terminal to run the bats tests The suite ran under `docker run -t`, which allocated a TTY, so it could take one for granted. On the runner there is none, and `$TERM` is unset: tput: No value for $TERM and no -T specified validator.bash: line 8: printf: write error: Broken pipe Two things assumed a terminal. `setup_file` printed a header through `tput`, which fails outright without `$TERM`, and the make target forced the pretty formatter, which needs one. `-t` and `-p` were both passed, which is contradictory since either sets the formatter, and the pretty one won. The header no longer goes through `tput`, and no formatter is forced, so bats picks the pretty one on a terminal and tap when there is none. Verified both ways, with `$TERM` set and unset: 10 tests pass, `make test-int` exits 0, and no tput or broken pipe errors in either. Co-Authored-By: Claude Opus 5 (1M context) --- Makefile | 3 ++- tests/bats/leverage.bats | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ee773f43..97ed3538 100644 --- a/Makefile +++ b/Makefile @@ -28,8 +28,9 @@ test-unit: ## Run unit tests and create a coverage report test-unit-no-cov: ## Run unit tests with no coverage report pytest --verbose --no-cov +# No formatter is forced: bats picks the pretty one on a terminal, and tap when there is none test-int: ## Run integration tests (requires bats, terraform and tofu, see README) - bats --verbose-run --show-output-of-passing-tests --print-output-on-failure -T -t -p -r tests/bats + bats --verbose-run --show-output-of-passing-tests --print-output-on-failure -T -r tests/bats tests: test-unit-no-cov test-int ## Run full set of tests diff --git a/tests/bats/leverage.bats b/tests/bats/leverage.bats index 4f4a94f2..c1ad0d63 100644 --- a/tests/bats/leverage.bats +++ b/tests/bats/leverage.bats @@ -1,5 +1,6 @@ setup_file(){ - echo "$(tput bold)========================== bats tests session starts ===========================" >&3 + # No `tput` here: it needs a terminal, and there is none when running on CI + echo "========================== bats tests session starts ===========================" >&3 } setup(){