pynol.environment¶

domain¶

Domain¶

class pynol.environment.domain.Domain(dimension)[source]¶

Bases: ABC

An abstract class representing the feasible domain.

Parameters:: dimension (int) – Dimension of the feasible set.

abstract init_x(prior: str | ndarray | None, seed: int | None) → ndarray[source]¶: Initialize a decision in the domain.

abstract project(x: ndarray)[source]¶: Project the decision \(x\) back to the feasible set.

Ball¶

class pynol.environment.domain.Ball(dimension: int, radius: float = 1.0, center: ndarray | None = None)[source]¶

Bases: Domain

This class defines a Euclid ball as the feasible set.

Parameters:

dimension (int) – Dimension of the feasible set.
radius (float) – Radius of the ball.
center (numpy.ndarray, optional) – Coordinates of the center point.
specified. (Default to the origin point if not) –

R¶

Radius of the minimum outside ball, which is useful for irregular domains.

Type:: float

r¶

Radius of the maximum inside ball, which is useful for irregular domains.

Type:: float

init_x(prior: str | ndarray | None, seed: int | None) → ndarray[source]¶

Initialize a decision in the domain.

Parameters:

prior (numpy.ndarray, optional) – Prior information to initialize the decision. If a numpy.ndarray is given, the method will return prior as the decision, otherwise return a random vector in the Ball.
seed (int, optional) – Random seed to initial the decision if prior=None.

Returns:

a decision in the ball.

Return type:

numpy.ndarray

project(x: ndarray) → ndarray[source]¶

Project the decision \(x\) back to the ball by Euclid distance.

Parameters:: x (numpy.ndarray) – the vector to be projected.
Returns:: the projected vector.
Return type:: numpy.ndarray

unit_vec(seed: int | None = None) → ndarray[source]¶

Sample a unit vector uniformly at random.

Parameters:: seed (int, optional) – Random seed to sample the vector.
Returns:: a decision in the ball.
Return type:: numpy.ndarray

Simplex¶

class pynol.environment.domain.Simplex(dimension: int)[source]¶

Bases: Domain

This class defines a simplex as the feasible set.

Parameters:: dimension (int) – Dimension of the feasible set.

init_x(prior: str | ndarray = 'uniform', seed: int | None = None) → ndarray[source]¶

Initialize a decision x in the domain.

Parameters:

prior (numpy.ndarray, 'uniform', 'nonuniform', optional) – Prior information to initialize the decision. If a numpy.ndarray is given, the method will return prior as the decision; if prior='uniform', the method will return the uniform vector \(x_i = 1/d, \forall i \in [d]\); if prior='nonuniform', the method will return \(x_i = \frac{d+1}{d} \cdot \frac{1}{i(i+1)}, \forall i \in [d]\), where \(d\) is the dimension of the simplex; if prior=None, the method will return a random vector in the simplex.
seed (int, optional) – Random seed to initial the decision if prior=None.

Returns:

a decision in the ball.

Return type:

numpy.ndarray

project(x: ndarray, dist: str = 'kl_div', norm: int | str = 1)[source]¶

Project the decision \(x\) back to the simplex.

Parameters:

x (numpy.ndarray) – Vector to be projected.
dist (str) – Distance metric used to project the decision. Valid options include 'kl_div' or 'norm'. if dist=kl_div, the return decision will be \(x_i = x_i / \sum_j x_j\), otherwise, the norm distance will be used to project the decision.
norm (int, str, optional) – Type of norm which is only used when dist='norm'. Valid options include any positive integer or 'inf' (infinity norm).

Returns:

the projected vector.

Return type:

numpy.ndarray

loss function¶

LossFunction¶

class pynol.environment.loss_function.LossFunction[source]¶

Bases: ABC

An abstract class for loss function.

Users can define their loss functions by inheriting from this class and override the method __getitem__().

InnerLoss¶

class pynol.environment.loss_function.InnerLoss(feature: ndarray | None = None, scale: float = 1.0)[source]¶

Bases: LossFunction

This class defines the inner loss function.

Parameters:

feature (numpy.ndarray) – Features of the environment.
scale (float) – Scale coefficient of the loss function.

Example:

import numpy as np
func = InnerLoss(feature=np.random.rand(1000, 5))  # 1000 rounds, 5 dimension

Then, call func[t] will return the inner loss function \(f_t(x) = \langle \varphi_t, x \rangle\), where \(\varphi_t\) is the feature at round \(t\).

SquareLoss¶

class pynol.environment.loss_function.SquareLoss(feature: ndarray | None = None, label: ndarray | None = None, scale: float = 1.0)[source]¶

Bases: LossFunction

This class defines the logistic loss function.

Parameters:

feature (numpy.ndarray) – Features of the environment.
label (numpy.ndarray) – Labels of the environment.
scale (float) – Scale coefficient of the loss function.

Example:

import numpy as np
feature, label = np.random.rand(1000, 5), np.random.randint(2, size=1000)
func = SquareLoss(feature, label)  # 1000 rounds, 5 dimension

Then, call func[t] will return the square loss function \(f_t(x) = \frac{1}{2} (y_t - \langle \varphi_t, x \rangle)^2\), where \(\varphi_t\) and \(y_t\) are the feature and label at round \(t\).

LogisticLoss¶

class pynol.environment.loss_function.LogisticLoss(feature: ndarray | None = None, label: ndarray | None = None, scale: float = 1.0)[source]¶

Bases: LossFunction

This class defines the logistic loss function.

Parameters:

Feature (numpy.ndarray) – Features of the environment.
label (numpy.ndarray) – Labels of the environment.
scale (float) – Scale coefficient of the loss function.

Example:

import numpy as np
feature, label = np.random.rand(1000, 5), np.random.randint(2, size=1000)
func = LogisticLoss(feature, label)  # 1000 rounds, 5 dimension

Then, call func[t] will return the loss function \(f_t(x) = y \log (\frac{1}{1+e^{-\varphi_t^\top x}})+(1-y) \log (1-\frac{1}{1+e^{-\varphi_t^\top x}})\) where \(\varphi_t\) and \(y_t\) are the feature and label at round \(t\).

FuncWithSwitch¶

class pynol.environment.loss_function.FuncWithSwitch(f: LossFunction | None = None, penalty: float = 1.0, norm: int = 2, order: int = 2)[source]¶

Bases: object

This class defines the loss function with switching cost.

Parameters:

f (LossFunction) – Origin loss function.
penalty (float) – Penalty coefficient of the switching cost.
norm (non-zero int, numpy.inf) – Order of the norm. The default is 2 norm.
order (int) – Order the the switching cost. The default is 2.

Example:

import numpy as np
feature, label = np.random.rand(1000, 5), np.random.randint(2, size=1000)
f = SquareLoss(feature, label)
func = FuncWithSwitch(f, penalty=1, norm=2, order=2)

Then, call func[t] will return the square loss function with switching cost \(f_t(x) = \frac{1}{2} (y_t - \langle \varphi_t, x \rangle)^2 + \lVert x - x_{t-1}\rVert_2^2\) where \(\varphi_t\) and \(y_t\) are the feature and label at round \(t\).

environment¶

Environment¶

class pynol.environment.environment.Environment(func_sequence: Iterable | None = None, optimism: ndarray | None = None, func: Callable[[ndarray], float] | None = None, grad: ndarray | None = None, grad_func: Callable[[ndarray], float] | None = None, surrogate_func: Callable[[ndarray], float] | None = None, surrogate_grad: ndarray | None = None, surrogate_grad_func: Callable[[ndarray], float] | None = None, use_surrogate_grad: bool = True, full_info: bool = True)[source]¶

Bases: object

Class for the environment, including loss function, optimism, and so on.

At each round, the environment can set two loss functions, one is the origin loss function func \(f_t\) and the other is the surrogate loss function surrogate_func \(f'_t\) (if any). The gradient of which function is given to the learner is determined by use_surrogate_grad.

Parameters:

func_sequence (Iterable, optional) – Loss function sequence for whole time horizon.
optimism (numpy.ndarray) – Optimism at the beginning of current round.
func (Callable, optional) – Origin loss function at current round.
grad (numpy.ndarray, optional) – Gradient of all decisions for origin loss function, only used when the gradient of all decisions are the same, namely, the liner function.
grad_func (Callable, optional) – Gradient function for origin loss function. It can be given by the environment to accelerate the running time. If it is not given, the origin gradient function will be computed by autograd.
surrogate_func (Callable, optional) – Surrogate loss function at current round.
surrogate_grad (numpy.ndarray, optional) – Gradient of all decisions for surrogate loss function, only used when the gradient of all decisions are the same, namely, the liner function.
surrogate_grad_func (Callable, optional) – Gradient function for surrogate loss function. It can be given by the environment to accelerate the running time. If it is not given, the surrogate gradient function will be computed by autograd.
use_surrogate_grad (bool) – Gradient of which function is returned by the environment.
full_info (bool) – Specify the type of feedback: full-information or bandit feedback.

get_grad(x: ndarray)[source]¶

Get the gradient of the decision \(x\).

Parameters:: x (numpy.ndarray) – Decision of the learner.
Returns:: Gradient of the decision \(x\).
Return type:: numpy.ndarray

get_loss(x: ndarray)[source]¶

Get the loss value of the decision \(x\).

Parameters:

x (numpy.ndarray) – Decision of the learner.

Returns:

tuple contains:

loss (float): Origin loss value.

surrogate_loss (float): Surrogate loss value

Return type:

tuple