teras.layers.SAINTMultiHeadInterSampleAttention

class teras.layers.SAINTMultiHeadInterSampleAttention(num_heads, key_dim, value_dim=None, dropout=0.0, **kwargs)

Multi Head Inter Sample Attention layer based on the SAINT architecture proposed in the “SAINT: Improved Neural Networks for Tabular Data” paper.

Reference(s):

https://arxiv.org/abs/2106.01342

Parameters:

• num_heads (int) – int, number of attention heads to use.

• key_dim (int) – int, the paper proposes to use embedding_dim/num_heads dimensions for your key dimensionality

• value_dim (int) – int, same value as key_dim is used by the paper.

• dropout (float) – float, dropout value to use. Defaults to 0.

Shapes:

Input Shape: (batch_size, num_features, embedding_dim) Output Shape: (batch_size, num_features, embedding_dim)

__init__(num_heads, key_dim, value_dim=None, dropout=0.0, **kwargs)

Methods

__init__(num_heads, key_dim[, value_dim, ...])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric()
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs)
compute_mask(inputs, previous_mask)
compute_output_shape(input_shape)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Creates an operation from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns the config of the object.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
load_own_variables(store)	Loads the state of the layer.
quantize(mode)
quantized_call(*args, **kwargs)
save_own_variables(store)	Saves the state of the layer.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.