There is only one ppc64 machine with the `v8test` label

[targos]

https://ci.nodejs.org/label/v8test/

It's https://ci.nodejs.org/computer/test-osuosl-centos7-ppc64_le-1/
Is there a reason for not having https://ci.nodejs.org/computer/test-osuosl-centos7-ppc64_le-2/ too?

[richardlau]

The only reason I can think of is to not block ppc64 behind V8 jobs, but I don't know we run them often enough for that to be a problem. I'd be up for adding the v8test label to the second machine -- we can remove the label again if we start seeing a large backlog. Thoughts @AshCripps @mhdawson?

[AshCripps]

No opposition from me.

[mhdawson]

I think that might be because I'd only manually installed gn, etc. on one of them at one point. Since I think richard has fixed that up no objections from me.

[richardlau]

Added the label to the second machine.

[richardlau]

Reopening as the second machine is failing V8 CI builds. Most likely due to a missing package(s).

[richardlau]

Ran the following to check the packages installed on both machines

$ ssh test-osuosl-centos7-ppc64_le-1 yum list installed > 1.txt
$ ssh test-osuosl-centos7-ppc64_le-2 yum list installed > 2.txt
$ diff -u 1.txt 2.txt

glib2-devel is on -1 but not -2.

Details

--- 1.txt       2021-07-13 07:27:09.642637462 -0400
+++ 2.txt       2021-07-13 07:27:17.398637525 -0400
@@ -169,14 +169,12 @@
 gdbm.ppc64le                          1.10-8.el7                 @anaconda
 gdisk.ppc64le                         0.8.10-3.el7               @base
 gdk-pixbuf2.ppc64le                   2.36.12-3.el7              @base
-geoipupdate.ppc64le                   2.5.0-1.el7                installed
+geoipupdate.ppc64le                   2.5.0-1.el7                @base
 gettext.ppc64le                       0.19.8.1-3.el7             @base
 gettext-libs.ppc64le                  0.19.8.1-3.el7             @base
 git.ppc64le                           1.8.3.1-23.el7_8           @updates
 glib2.ppc64le                         2.56.1-8.el7               @updates
 glib2.ppc64le                         2.56.1-9.el7_9             installed
-glib2-devel.ppc64le                   2.56.1-8.el7               @updates
-glib2-devel.ppc64le                   2.56.1-9.el7_9             installed
 glibc.ppc64le                         2.17-323.el7_9             @updates
 glibc.ppc64le                         2.17-324.el7_9             installed
 glibc-common.ppc64le                  2.17-323.el7_9             @updates
@@ -211,7 +209,7 @@
 hardlink.ppc64le                      1:1.0-19.el7               @anaconda
 harfbuzz.ppc64le                      1.7.5-2.el7                @base
 hicolor-icon-theme.noarch             0.12-7.el7                 @base
-hostname.ppc64le                      3.13-3.el7_7.1             @base
+hostname.ppc64le                      3.13-3.el7_7.1             @updates
 hwdata.ppc64le                        0.252-9.7.el7              @base
 info.ppc64le                          5.1-5.el7                  @anaconda
 initscripts.ppc64le                   9.49.53-1.el7_9.1          @updates
@@ -230,13 +228,13 @@
 kbd.ppc64le                           1.15.5-15.el7              @anaconda
 kbd-legacy.noarch                     1.15.5-15.el7              @anaconda
 kbd-misc.noarch                       1.15.5-15.el7              @anaconda
-kernel.ppc64le                        3.10.0-1127.el7            installed
+kernel.ppc64le                        3.10.0-1127.el7            @base
 kernel.ppc64le                        3.10.0-1127.13.1.el7       @updates
 kernel.ppc64le                        3.10.0-1127.18.2.el7       @updates
 kernel.ppc64le                        3.10.0-1160.11.1.el7       @updates
 kernel.ppc64le                        3.10.0-1160.21.1.el7       @updates
 kernel.ppc64le                        3.10.0-1160.31.1.el7       installed
-kernel-devel.ppc64le                  3.10.0-1127.el7            installed
+kernel-devel.ppc64le                  3.10.0-1127.el7            @base
 kernel-devel.ppc64le                  3.10.0-1127.13.1.el7       @updates
 kernel-devel.ppc64le                  3.10.0-1127.18.2.el7       @updates
 kernel-devel.ppc64le                  3.10.0-1160.11.1.el7       @updates
@@ -417,7 +415,7 @@
 numactl-libs.ppc64le                  2.0.12-5.el7               @base
 openldap.ppc64le                      2.4.44-22.el7              @base
 openldap.ppc64le                      2.4.44-23.el7_9            installed
-openssh.ppc64le                       7.4p1-21.el7               installed
+openssh.ppc64le                       7.4p1-21.el7               @base
 openssh-clients.ppc64le               7.4p1-21.el7               @base
 openssh-server.ppc64le                7.4p1-21.el7               @base
 openssl.ppc64le                       1:1.0.2k-21.el7_9          @updates
@@ -566,7 +564,7 @@
 python3.ppc64le                       3.6.8-18.el7               @updates
 python3-libs.ppc64le                  3.6.8-18.el7               @updates
 python3-pip.noarch                    9.0.3-8.el7                @base
-python3-setuptools.noarch             39.2.0-10.el7              installed
+python3-setuptools.noarch             39.2.0-10.el7              @base
 pyxattr.ppc64le                       0.5.1-5.el7                @anaconda
 qemu-guest-agent.ppc64le              10:2.12.0-3.el7            @base
 qemu-img-ma.ppc64le                   10:2.12.0-48.el7           @base

Example failing V8 CI:
https://ci.nodejs.org/job/node-test-commit-v8-linux/4134/nodes=centos7-ppcle,v8test=v8test/console

07:49:02 ERROR at //build/config/linux/pkg_config.gni:103:17: Script returned non-zero exit code.
07:49:02     pkgresult = exec_script(pkg_config_script, args, "value")
07:49:02                 ^----------
07:49:02 Current dir: /home/iojs/build/workspace/node-test-commit-v8-linux/deps/v8/out.gn/ppc64.release/
07:49:02 Command: python /home/iojs/build/workspace/node-test-commit-v8-linux/deps/v8/build/config/linux/pkg-config.py glib-2.0 gmodule-2.0 gobject-2.0 gthread-2.0
07:49:02 Returned 1.
07:49:02 stderr:
07:49:02 
07:49:02 Package glib-2.0 was not found in the pkg-config search path.
07:49:02 Perhaps you should add the directory containing `glib-2.0.pc'
07:49:02 to the PKG_CONFIG_PATH environment variable
07:49:02 No package 'glib-2.0' found
07:49:02 Package gmodule-2.0 was not found in the pkg-config search path.
07:49:02 Perhaps you should add the directory containing `gmodule-2.0.pc'
07:49:02 to the PKG_CONFIG_PATH environment variable
07:49:02 No package 'gmodule-2.0' found
07:49:02 Package gobject-2.0 was not found in the pkg-config search path.
07:49:02 Perhaps you should add the directory containing `gobject-2.0.pc'
07:49:02 to the PKG_CONFIG_PATH environment variable
07:49:02 No package 'gobject-2.0' found
07:49:02 Package gthread-2.0 was not found in the pkg-config search path.
07:49:02 Perhaps you should add the directory containing `gthread-2.0.pc'
07:49:02 to the PKG_CONFIG_PATH environment variable
07:49:02 No package 'gthread-2.0' found
07:49:02 Could not run pkg-config.
07:49:02 
07:49:02 See //build/config/linux/BUILD.gn:47:3: whence it was called.
07:49:02   pkg_config("glib") {
07:49:02   ^-------------------
07:49:02 See //build/config/compiler/BUILD.gn:258:18: which caused the file to be included.
07:49:02     configs += [ "//build/config/linux:compiler" ]
07:49:02                  ^------------------------------
07:49:02 Loading //test/mjsunit/BUILD.gn (referenced from //test/BUILD.gn:15)
07:49:02 make: *** [Makefile:274: v8] Error 1

$ ssh test-osuosl-centos7-ppc64_le-1 pkg-config --list-all | sort > 1-pkg-config.txt
$ ssh test-osuosl-centos7-ppc64_le-2 pkg-config --list-all | sort > 2-pkg-config.txt
$ diff -u --ignore-space-change 1-pkg-config.txt 2-pkg-config.txt

--- 1-pkg-config.txt    2021-07-13 09:16:49.724690930 -0400
+++ 2-pkg-config.txt    2021-07-13 09:16:57.489690993 -0400
@@ -1,14 +1,6 @@
 com_err               com_err - Common error description library
 dracut                dracut - dracut
-gio-2.0               GIO - glib I/O library
-gio-unix-2.0          GIO unix specific APIs - unix specific headers for glib I/O library
-glib-2.0              GLib - C Utility Library
-gmodule-2.0           GModule - Dynamic module loader for GLib
-gmodule-export-2.0    GModule - Dynamic module loader for GLib
-gmodule-no-export-2.0 GModule - Dynamic module loader for GLib
-gobject-2.0           GObject - GLib Type, Object, Parameter and Signal Library
 gssrpc                gssrpc - GSSAPI RPC implementation
-gthread-2.0           GThread - Thread support for GLib
 kadm-client           kadm-client - Kerberos administration client library
 kadm-server           kadm-server - Kerberos administration server library
 kdb                   kdb - Kerberos database access libraries

[richardlau]

PR to install glib2-devel via Ansible on the ppc64 machines: #2709