Skip to content

Use the built-in benchmark functions¤

In this example, we will use the built-in benchmark functions provided by the f3dasm.datageneration submodule to generate output for a data-driven experiment.

The f3dasm framework includes a collection of benchmark functions designed for testing the performance of optimization algorithms or simulating expensive computations to evaluate the data-driven process. These functions are adapted from the Python Benchmark Test Optimization Function Single Objective GitHub repository.

The built-in benchmark functions take a single array-valued input x, so we'll build a domain with one ArrayParameter rather than separate float dimensions.

from f3dasm.design import Domain

domain = Domain()
domain.add_array(name="x", shape=(2,), low=-1.0, high=1.0)
domain.add_output("y")

We generate the input data by sampling the domain with a grid sampler and create the ExperimentData object:

from f3dasm import ExperimentData, create_sampler

experiment_data = ExperimentData(domain=domain)
sampler = create_sampler("random", seed=42)

experiment_data = sampler.call(data=experiment_data, n_samples=20)

experiment_data

Evaluating a 2D version of the 'Ackley' function can be done in two ways:

Method 1: Providing a function name as a string to the create_datagenerator function¤

Import create_datagenerator and pass the function name plus the output_names. This returns a DataGenerator block that can be armed and called:

from f3dasm import create_datagenerator

data_generator = create_datagenerator(
    "Ackley",
    output_names="y",
    scale_bounds=[[-1.0, 1.0], [-1.0, 1.0]],
    offset=False,
)

data_generator.arm(experiment_data)
experiment_data = data_generator.call(data=experiment_data)

experiment_data

Method 2: Wrapping the raw function with @datagenerator¤

The functions under f3dasm.datageneration.functions are plain numpy functions. Wrap one with the @datagenerator decorator to get a DataGenerator block:

import numpy as np

from f3dasm import datagenerator
from f3dasm.datageneration.functions import ackley


@datagenerator(output_names="y")
def ackley_block(x: np.ndarray) -> float:
    return float(ackley(x))


# Fresh experiment_data because the previous one has already been evaluated
experiment_data2 = ExperimentData(domain=domain)
experiment_data2 = sampler.call(data=experiment_data2, n_samples=20)

ackley_block.arm(experiment_data2)
experiment_data2 = ackley_block.call(data=experiment_data2)

experiment_data2

Common optional keyword arguments for create_datagenerator when using built-ins:

  • scale_bounds: a 2D list of floats that define the scaling lower and upper boundaries for each dimension. The benchmark function's box-constraints are scaled to these boundaries.
  • noise: standard deviation of Gaussian noise added to the objective value.
  • offset: if True, offsets the benchmark function by a random constant vector so the minimum is not at the origin (useful for testing optimizer robustness).
  • seed: seed for the noise and offset random number generator.
import matplotlib.pyplot as plt

arr_in, arr_out = experiment_data.to_numpy()
x_arr = np.stack(arr_in[:, 0])
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.scatter(x_arr[:, 0], x_arr[:, 1], arr_out.ravel())
_ = ax.set_xlabel("$x_0$")
_ = ax.set_ylabel("$x_1$")
_ = ax.set_zlabel("$f(x)$")

Next: Built-in Optimizers