Benchmark Repository for SLOPE

This repository is dedicated to regression with the Sorted L-One Penalized Estimation (SLOPE) estimator which consists in solving the following program:

$$ \min_{\beta} \, \tfrac{1}{2n} \Vert y - X\beta \Vert^2_2 + J(\beta, \lambda) $$

where

$$ J(\beta, \lambda) = \sum_{j=1}^p \lambda_j | \beta_{(j)}| $$

with $\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_p$ and $|\beta_{(1)}| \geq |\beta_{(2)}| \geq ... \geq |\beta_{(p)}|$.

We note $n$ (or n_samples) the number of samples and $p$ (or n_features) the number of features. We also have that $X\in \mathbb{R}^{n\times p}$ and $y\in \mathbb{R}^n$.

Installing Benchopt

This benchmark relies on benchopt, a generic framework for running numerical benchmarks. The recommended way to use benchopt is within a conda environment. So, begin by creating and activating a new conda environment and install benchopt in it:

$ conda create -n benchopt python
$ conda activate benchopt
$ pip install -U benchopt

NixOS

Conda packages cannot normally run directly on NixOS. The included devenv configuration supplies a compatible FHS environment while leaving dependency management to Benchopt. Enter the shell and install Benchopt once:

$ devenv shell
$ benchopt-setup

The benchopt wrapper then runs commands inside the Conda environment. The benchmark's Conda base environment lives under ~/.local/state/devenv/benchmark_slope/conda. It must remain outside the checkout because Benchopt 1.9.1 misidentifies checkout-local installations when creating test environments. Conda's package cache and secondary environments remain in devenv's ignored state directory.

Installing the Benchmark

To install the benchmark, clone this repository and move to its folder:

$ git clone https://github.com/benchopt/benchmark_slope
$ cd benchmark_slope/

To install the dependencies for the solvers and datasets for the benchmark, first make sure that you have activated the conda environment where benchopt is installed. Then, you can either install all the dependencies with:

$ benchopt install .

Or you can install only a subset of solvers by specifying them with the -s option.

Running the Benchmark

To run the benchmark, simply use the benchopt run command:

$ benchopt run .

By default, all solvers and datasets are run. You can restrict the benchmark to some solvers or datasets, e.g.:

$ benchopt run -s PGD[prox=prox_fast_stack] -d libsvm[dataset=real-sim,standardize=True]

You can also specify a YAML configuration file to set the parameters of the benchmark. An example config is provided in <example_config.yml>.

$ benchopt run --config example_config.yml .

Use benchopt run -h for more details about these options, or visit https://benchopt.github.io/api.html.

Accuracy target

The benchmark stops sampling a solver's convergence curve once its measured relative duality gap reaches target_rel_duality_gap. The default target is 1e-7. This objective-level rule applies uniformly to all solvers and avoids requesting accuracy beyond the range used for comparison. Set the objective parameter explicitly in a configuration file to use a different target.

Acknowledgments

This repository is based on the work of Johan Larsson, Quentin Klopfenstein, Mathurin Massias and Jonas Wallin at https://github.com/Klopfe/benchmark_slope.