foolbox.models

Provides classes to wrap existing models in different framworks so that they provide a unified API to the attacks.

Models

Model Base class to provide attacks with a unified interface to models.
DifferentiableModel Base class for differentiable models.
TensorFlowModel Creates a Model instance from existing TensorFlow tensors.
TensorFlowEagerModel Creates a Model instance from a TensorFlow model using eager execution.
PyTorchModel Creates a Model instance from a PyTorch module.
KerasModel Creates a Model instance from a Keras model.
TheanoModel Creates a Model instance from existing Theano tensors.
LasagneModel Creates a Model instance from a Lasagne network.
MXNetModel Creates a Model instance from existing MXNet symbols and weights.
MXNetGluonModel Creates a Model instance from an existing MXNet Gluon Block.
JAXModel Creates a Model instance from a JAX predict function.
CaffeModel

Wrappers

ModelWrapper Base class for models that wrap other models.
DifferentiableModelWrapper Base class for models that wrap other models and provide gradient methods.
ModelWithoutGradients Turns a model into a model without gradients.
ModelWithEstimatedGradients Turns a model into a model with gradients estimated by the given gradient estimator.
CompositeModel Combines predictions of a (black-box) model with the gradient of a (substitute) model.

Detailed description

class foolbox.models.Model(bounds, channel_axis, preprocessing=(0, 1))[source]

Base class to provide attacks with a unified interface to models.

The Model class represents a model and provides a unified interface to its predictions. Subclasses must implement forward and num_classes.

Model instances can be used as context managers and subclasses can require this to allocate and release resources.

Parameters:
bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_one(self, x)[source]

Takes a single input and returns the logits predicted by the underlying model.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

Returns:
numpy.ndarray

Predicted logits with shape (number of classes,).

See also

forward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.DifferentiableModel(bounds, channel_axis, preprocessing=(0, 1))[source]

Base class for differentiable models.

The DifferentiableModel class can be used as a base class for models that can support gradient backpropagation. Subclasses must implement gradient and backward.

A differentiable model does not necessarily provide reasonable values for the gradient, the gradient can be wrong. It only guarantees that the relevant methods can be called.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

backward_one(self, gradient, x)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the input.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (number of classes,).

x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the input.

See also

backward()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

gradient_one(self, x, label)[source]

Takes a single input and label and returns the gradient of the cross-entropy loss w.r.t. the input.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

gradient()
class foolbox.models.TensorFlowModel(inputs, logits, bounds, channel_axis=3, preprocessing=(0, 1))[source]

Creates a Model instance from existing TensorFlow tensors.

Parameters:
inputs : tensorflow.Tensor

The input to the model, usually a tensorflow.placeholder.

logits : tensorflow.Tensor

The predictions of the model, before the softmax.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
classmethod from_keras(model, bounds, input_shape=None, channel_axis='auto', preprocessing=(0, 1))[source]

Alternative constructor for a TensorFlowModel that accepts a tf.keras.Model instance.

Parameters:
model : tensorflow.keras.Model

A tensorflow.keras.Model that accepts a single input tensor and returns a single output tensor representing logits.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

input_shape : tuple

The shape of a single input, e.g. (28, 28, 1) for MNIST. If None, tries to get the the shape from the model’s input_shape attribute.

channel_axis : int or ‘auto’

The index of the axis that represents color channels. If ‘auto’, will be set automatically based on keras.backend.image_data_format()

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.TensorFlowEagerModel(model, bounds, num_classes=None, channel_axis=3, preprocessing=(0, 1))[source]

Creates a Model instance from a TensorFlow model using eager execution.

Parameters:
model : a TensorFlow eager model

The TensorFlow eager model that should be attacked. It will be called with input tensors and should return logits.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

num_classes : int

If None, will try to infer it from the model’s output shape.

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.PyTorchModel(model, bounds, num_classes, channel_axis=1, device=None, preprocessing=(0, 1))[source]

Creates a Model instance from a PyTorch module.

Parameters:
model : torch.nn.Module

The PyTorch model that should be attacked. It should predict logits or log-probabilities, i.e. predictions without the softmax.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

num_classes : int

Number of classes for which the model will output predictions.

channel_axis : int

The index of the axis that represents color channels.

device : string

A string specifying the device to do computation on. If None, will default to “cuda:0” if torch.cuda.is_available() or “cpu” if not.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.JAXModel(predict, bounds, num_classes, channel_axis=3, preprocessing=(0, 1))[source]

Creates a Model instance from a JAX predict function.

Parameters:
predict : function

The JAX-compatible function that takes a batch of inputs as and returns a batch of predictions (logits); use functools.partial(predict, params) to pass params if necessary

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

num_classes : int

Number of classes for which the model will output predictions.

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.KerasModel(model, bounds, channel_axis='auto', preprocessing=(0, 1), predicts='probabilities')[source]

Creates a Model instance from a Keras model.

Parameters:
model : keras.models.Model

The Keras model that should be attacked.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int or ‘auto’

The index of the axis that represents color channels. If ‘auto’, will be set automatically based on keras.backend.image_data_format()

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

predicts : str

Specifies whether the Keras model predicts logits or probabilities. Logits are preferred, but probabilities are the default.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.TheanoModel(inputs, logits, bounds, num_classes, channel_axis=1, preprocessing=[0, 1])[source]

Creates a Model instance from existing Theano tensors.

Parameters:
inputs : theano.tensor

The input to the model.

logits : theano.tensor

The predictions of the model, before the softmax.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

num_classes : int

Number of classes for which the model will output predictions.

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.LasagneModel(input_layer, logits_layer, bounds, channel_axis=1, preprocessing=(0, 1))[source]

Creates a Model instance from a Lasagne network.

Parameters:
input_layer : lasagne.layers.Layer

The input to the model.

logits_layer : lasagne.layers.Layer

The output of the model, before the softmax.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

class foolbox.models.MXNetModel(data, logits, args, ctx, num_classes, bounds, channel_axis=1, aux_states=None, preprocessing=(0, 1))[source]

Creates a Model instance from existing MXNet symbols and weights.

Parameters:
data : mxnet.symbol.Variable

The input to the model.

logits : mxnet.symbol.Symbol

The predictions of the model, before the softmax.

args : dictionary mapping str to mxnet.nd.array

The parameters of the model.

ctx : mxnet.context.Context

The device, e.g. mxnet.cpu() or mxnet.gpu().

num_classes : int

The number of classes.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int

The index of the axis that represents color channels.

aux_states : dictionary mapping str to mxnet.nd.array

The states of auxiliary parameters of the model.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.MXNetGluonModel(block, bounds, num_classes, ctx=None, channel_axis=1, preprocessing=(0, 1))[source]

Creates a Model instance from an existing MXNet Gluon Block.

Parameters:
block : mxnet.gluon.Block

The Gluon Block representing the model to be run.

ctx : mxnet.context.Context

The device, e.g. mxnet.cpu() or mxnet.gpu().

num_classes : int

The number of classes.

bounds : tuple

Tuple of lower and upper bound for the pixel values, usually (0, 1) or (0, 255).

channel_axis : int

The index of the axis that represents color channels.

preprocessing: dict or tuple

Can be a tuple with two elements representing mean and standard deviation or a dict with keys “mean” and “std”. The two elements should be floats or numpy arrays. “mean” is subtracted from the input, the result is then divided by “std”. If “mean” and “std” are 1-dimensional arrays, an additional (negative) “axis” key can be given such that “mean” and “std” will be broadcasted to that axis (typically -1 for “channels_last” and -3 for “channels_first”, but might be different when using e.g. 1D convolutions). Finally, a (negative) “flip_axis” can be specified. This axis will be flipped (before “mean” is subtracted), e.g. to convert RGB to BGR.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.CaffeModel(net, bounds, channel_axis=1, preprocessing=(0, 1), data_blob_name='data', label_blob_name='label', output_blob_name='output')[source]
backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.ModelWrapper(model)[source]

Base class for models that wrap other models.

This base class can be used to implement model wrappers that turn models into new models, for example by preprocessing the input or modifying the gradient.

Parameters:
model : Model

The model that is wrapped.

forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.

class foolbox.models.DifferentiableModelWrapper(model)[source]

Base class for models that wrap other models and provide gradient methods.

This base class can be used to implement model wrappers that turn models into new models, for example by preprocessing the input or modifying the gradient.

Parameters:
model : Model

The model that is wrapped.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward_and_gradient(self, x, label)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
class foolbox.models.ModelWithoutGradients(model)[source]

Turns a model into a model without gradients.

class foolbox.models.ModelWithEstimatedGradients(model, gradient_estimator)[source]

Turns a model into a model with gradients estimated by the given gradient estimator.

Parameters:
model : Model

The model that is wrapped.

gradient_estimator : callable

Callable taking three arguments (pred_fn, x, label) and returning the estimated gradients. pred_fn will be the forward method of the wrapped model.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
class foolbox.models.CompositeModel(forward_model, backward_model)[source]

Combines predictions of a (black-box) model with the gradient of a (substitute) model.

Parameters:
forward_model : Model

The model that should be fooled and will be used for predictions.

backward_model : Model

The model that provides the gradients.

backward(self, gradient, inputs)[source]

Backpropagates the gradient of some loss w.r.t. the logits through the underlying model and returns the gradient of that loss w.r.t to the inputs.

Parameters:
gradient : numpy.ndarray

Gradient of some loss w.r.t. the logits with shape (batch size, number of classes).

inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

The gradient of the respective loss w.r.t the inputs.

See also

backward_one()
gradient()
forward(self, inputs)[source]

Takes a batch of inputs and returns the logits predicted by the underlying model.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

See also

forward_one()
forward_and_gradient(self, inputs, labels)[source]

Takes inputs and labels and returns both the logits predicted by the underlying model and the gradients of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Inputs with shape as expected by the model (with the batch dimension).

labels : numpy.ndarray

Array of the class label of the inputs as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
forward_and_gradient_one(self, x, label)[source]

Takes a single input and label and returns both the logits predicted by the underlying model and the gradient of the cross-entropy loss w.r.t. the input.

Defaults to individual calls to forward_one and gradient_one but can be overriden by subclasses to provide a more efficient implementation.

Parameters:
x : numpy.ndarray

Single input with shape as expected by the model (without the batch dimension).

label : int

Class label of the input as an integer in [0, number of classes).

Returns:
numpy.ndarray

Predicted logits with shape (batch size, number of classes).

numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the input.

See also

forward_one()
gradient_one()
gradient(self, inputs, labels)[source]

Takes a batch of inputs and labels and returns the gradient of the cross-entropy loss w.r.t. the inputs.

Parameters:
inputs : numpy.ndarray

Batch of inputs with shape as expected by the underlying model.

labels : numpy.ndarray

Class labels of the inputs as a vector of integers in [0, number of classes).

Returns:
gradient : numpy.ndarray

The gradient of the cross-entropy loss w.r.t. the inputs.

See also

gradient_one()
backward()
num_classes(self)[source]

Determines the number of classes.

Returns:
int

The number of classes for which the model creates predictions.