teras.models.SAINTPretrainer

class teras.models.SAINTPretrainer(model, cardinalities, embedding_dim, cutmix_probability=0.3, mixup_alpha=1.0, temperature=0.7, lambda_=10.0, lambda_c=0.5, seed=1337, **kwargs)

SAINTPretrainer as proposed in the paper, “SAINT: Improved Neural Networks for Tabular Data”.

Reference(s):

https://arxiv.org/abs/2106.01342

Parameters:

• model (Model) – keras.Model, instance of SAINTBackbone model to pretrain

• cardinalities (list) – list, a list cardinalities of all the features in the dataset in the same order as the features’ occurrence. For numerical features, use the value 0 as indicator at the corresponding index. You can use the compute_cardinalities function from teras.utils package for this purpose.

• embedding_dim (int) – int, dimensionality of the embeddings being used in the model.

• cutmix_probability (float) – float, used by the CutMix layer in generation of mask that is used to mix samples together. Defaults to 0.3

• mixup_alpha (float) – float, used by the MixUp layer in sampling from the Beta distribution which is then used to interpolate samples. Defaults to 1.

• temperature (float) – float, used in the computation of the contrastive_loss to scale logits. Defaults to 0.7

• lambda – float, acts as a weight when adding the contrastive loss and the denoising loss together. loss = constrastive_loss + lambda_ * denoising_loss Defaults to 10.

• lambda_c (float) – float, used in the computation of the contrastive loss. Similar to lambda_ is helps combined two sub-losses within the contrastive loss. Defaults to 0.5

• seed (int) – int, seed used in random sampling and shuffling etc. It helps make the model behavior more deterministic. Defaults to (you guessed it) 1337.

__init__(model, cardinalities, embedding_dim, cutmix_probability=0.3, mixup_alpha=1.0, temperature=0.7, lambda_=10.0, lambda_c=0.5, seed=1337, **kwargs)

Methods

__init__(model, cardinalities, embedding_dim)
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric()
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs, **kwargs)
compile([optimizer, loss, loss_weights, ...])	Configures the model for training.
compile_from_config(config)	Compiles the model with the information given in config.
compiled_loss(y, y_pred[, sample_weight, ...])
compute_loss(x, x_reconstructed, z, ...[, ...])
compute_mask(inputs, previous_mask)
compute_metrics(x, y, y_pred[, sample_weight])	Update metric states and collect all metrics to be returned.
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
evaluate([x, y, batch_size, verbose, ...])	Returns the loss value & metrics values for the model in test mode.
export(filepath[, format])	Create a TF SavedModel artifact for inference.
fit([x, y, batch_size, epochs, verbose, ...])	Trains the model for a fixed number of epochs (dataset iterations).
from_config(config)	Creates an operation from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_compile_config()	Returns a serialized config with information for compiling the model.
get_config()	Returns the config of the object.
get_layer([name, index])	Retrieves a layer based on either its name (unique) or index.
get_metrics_result()	Returns the model's metrics values as a dict.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
load_own_variables(store)	Loads the state of the layer.
load_weights(filepath[, skip_mismatch])	Load weights from a file saved via save_weights().
loss(y, y_pred[, sample_weight])
make_predict_function([force])
make_test_function([force])
make_train_function([force])
predict(x[, batch_size, verbose, steps, ...])	Generates output predictions for the input samples.
predict_on_batch(x)	Returns predictions for a single batch of samples.
predict_step(data)
quantize(mode)	Quantize the weights of the model.
quantized_call(*args, **kwargs)
reset_metrics()
save(filepath[, overwrite, zipped])	Saves a model as a .keras file.
save_own_variables(store)	Saves the state of the layer.
save_weights(filepath[, overwrite])	Saves all layer weights to a .weights.h5 file.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
stateless_compute_loss(trainable_variables, ...)
summary([line_length, positions, print_fn, ...])	Prints a string summary of the network.
symbolic_call(*args, **kwargs)
test_on_batch(x[, y, sample_weight, return_dict])	Test the model on a single batch of samples.
test_step(data)
to_json(**kwargs)	Returns a JSON string containing the network configuration.
train_on_batch(x[, y, sample_weight, ...])	Runs a single gradient update on a single batch of data.
train_step(data)

Attributes

compiled_metrics
compute_dtype	The dtype of the computations performed by the layer.
distribute_reduction_method
distribute_strategy
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
jit_compile
layers
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_names
metrics_variables	List of all metric variables.
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
pretrained_embedding_layer
pretrained_model
quantization_mode	The quantization mode of this layer, None if not quantized.
run_eagerly
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.