teras.models.TabTransformerRTDPretrainer

class teras.models.TabTransformerRTDPretrainer(model, data_dim, replace_rate=0.3, mask_seed=1337, shuffle_seed=1999, **kwargs)

Replaced Token Detection (RTD) based Pretrainer for pretraining TabTransformerBackbone as proposed by Huang et al. in the paper, “TabTransformer: Tabular Data Modeling Using Contextual Embeddings”.

Reference(s):

https://arxiv.org/abs/2012.06678

Parameters:

• model (Model) – keras.Model, instance of TabTransformerBackbone to pretrain

• data_dim (int) – int, dimensionality of the input dataset

• replace_rate (float) – float, fraction of original features to replace. Must be in the range [0, 1). Defaults to 0.3 (or 30%)

• mask_seed (int) – int, seed for generating mask. Defaults to 1337

• shuffle_seed (int) – int, seed for shuffling inputs. Defaults to 1999

__init__(model, data_dim, replace_rate=0.3, mask_seed=1337, shuffle_seed=1999, **kwargs)

Methods

__init__(model, data_dim[, replace_rate, ...])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric()
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build(input_shape)
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs, mask, **kwargs)
compile([loss, optimizer])	Configures the model for training.
compile_from_config(config)	Compiles the model with the information given in config.
compiled_loss(y, y_pred[, sample_weight, ...])
compute_loss([x, y, y_pred, sample_weight, ...])	Compute the total loss, validate it, and return it.
compute_mask(inputs, previous_mask)
compute_metrics(x, y, y_pred[, sample_weight])	Update metric states and collect all metrics to be returned.
compute_output_shape(*args, **kwargs)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
evaluate([x, y, batch_size, verbose, ...])	Returns the loss value & metrics values for the model in test mode.
export(filepath[, format])	Create a TF SavedModel artifact for inference.
fit([x, y, batch_size, epochs, verbose, ...])	Trains the model for a fixed number of epochs (dataset iterations).
from_config(config)	Creates an operation from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_compile_config()	Returns a serialized config with information for compiling the model.
get_config()	Returns the config of the object.
get_layer([name, index])	Retrieves a layer based on either its name (unique) or index.
get_metrics_result()	Returns the model's metrics values as a dict.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
load_own_variables(store)	Loads the state of the layer.
load_weights(filepath[, skip_mismatch])	Load weights from a file saved via save_weights().
loss(y, y_pred[, sample_weight])
make_predict_function([force])
make_test_function([force])
make_train_function([force])
predict(x[, batch_size, verbose, steps, ...])	Generates output predictions for the input samples.
predict_on_batch(x)	Returns predictions for a single batch of samples.
predict_step(data)
quantize(mode)	Quantize the weights of the model.
quantized_call(*args, **kwargs)
reset_metrics()
save(filepath[, overwrite, zipped])	Saves a model as a .keras file.
save_own_variables(store)	Saves the state of the layer.
save_weights(filepath[, overwrite])	Saves all layer weights to a .weights.h5 file.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
stateless_compute_loss(trainable_variables, ...)
summary([line_length, positions, print_fn, ...])	Prints a string summary of the network.
symbolic_call(*args, **kwargs)
test_on_batch(x[, y, sample_weight, return_dict])	Test the model on a single batch of samples.
test_step(data)
to_json(**kwargs)	Returns a JSON string containing the network configuration.
train_on_batch(x[, y, sample_weight, ...])	Runs a single gradient update on a single batch of data.
train_step(data)

Attributes

compiled_metrics
compute_dtype	The dtype of the computations performed by the layer.
distribute_reduction_method
distribute_strategy
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
jit_compile
layers
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_names
metrics_variables	List of all metric variables.
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
pretrained_model
quantization_mode	The quantization mode of this layer, None if not quantized.
run_eagerly
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.