teras.layers.TabTransformerColumnEmbedding

class teras.layers.TabTransformerColumnEmbedding(cardinalities, embedding_dim, use_shared_embedding=True, shared_embedding_dim=None, join_method='concat', **kwargs)

Column Embedding layer as proposed in the “TabTransformer: Tabular Data Modeling Using Contextual Embeddings”.

Reference(s):

https://arxiv.org/abs/2012.06678

Parameters:

• cardinalities (list) – list, a list cardinalities of all the features in the dataset in the same order as the features’ occurrence. For numerical features, use any value <=0 as indicator at the corresponding index. You can use the compute_cardinalities function from teras.utils package for this purpose.

• embedding_dim (int) – int, dimensionality of the embeddings

• use_shared_embedding (bool) – bool, whether to use the shared embeddings. Defaults to True. If False, this layer will be effectively equivalent to a keras.layers.Embedding layer.

• shared_embedding_dim (int) – int, dimensionality of the shared embeddings. Shared embeddings are the embeddings of the unique column identifiers, which according to the paper, help the model distinguish categories of one feature from the other. By default, its value is set to embedding_dim / 8 as this setup is the most superior in the results presented by the authors.

• join_method (str) – str, one of [‘concat’, ‘add’] method to join the shared embeddings with feature embeddings. Defaults to ‘concat’, which is the recommended method, in which shared embeddings of shared_embedding_dim are concatenated with embedding_dim - shared_embedding_dim dimension feature embeddings. In ‘add’, shared embeddings have the same dimensions as the features, i.e. the embedding_dim and they are element-wise added to the features.

__init__(cardinalities, embedding_dim, use_shared_embedding=True, shared_embedding_dim=None, join_method='concat', **kwargs)

Methods

__init__(cardinalities, embedding_dim[, ...])
add_loss(loss)	Can be called inside of the call() method to add a scalar loss.
add_metric()
add_variable(shape, initializer[, dtype, ...])	Add a weight variable to the layer.
add_weight([shape, initializer, dtype, ...])	Add a weight variable to the layer.
build([input_shape])
build_from_config(config)	Builds the layer's states with the supplied config dict.
call(inputs)
compute_mask(inputs, previous_mask)
compute_output_shape(input_shape)
compute_output_spec(*args, **kwargs)
count_params()	Count the total number of scalars composing the weights.
from_config(config)	Creates an operation from its config.
get_build_config()	Returns a dictionary with the layer's input shape.
get_config()	Returns the config of the object.
get_weights()	Return the values of layer.weights as a list of NumPy arrays.
load_own_variables(store)	Loads the state of the layer.
quantize(mode)
quantized_call(*args, **kwargs)
save_own_variables(store)	Saves the state of the layer.
set_weights(weights)	Sets the values of layer.weights from a list of NumPy arrays.
stateless_call(trainable_variables, ...[, ...])	Call the layer without any side effects.
symbolic_call(*args, **kwargs)

Attributes

compute_dtype	The dtype of the computations performed by the layer.
dtype	Alias of layer.variable_dtype.
dtype_policy
input	Retrieves the input tensor(s) of a symbolic operation.
input_dtype	The dtype layer inputs should be converted to.
input_spec
losses	List of scalar losses from add_loss, regularizers and sublayers.
metrics	List of all metrics.
metrics_variables	List of all metric variables.
non_trainable_variables	List of all non-trainable layer state.
non_trainable_weights	List of all non-trainable weight variables of the layer.
output	Retrieves the output tensor(s) of a layer.
path	The path of the layer.
quantization_mode	The quantization mode of this layer, None if not quantized.
supports_masking	Whether this layer supports computing a mask using compute_mask.
trainable	Settable boolean, whether this layer should be trainable or not.
trainable_variables	List of all trainable layer state.
trainable_weights	List of all trainable weight variables of the layer.
variable_dtype	The dtype of the state (weights) of the layer.
variables	List of all layer state, including random seeds.
weights	List of all weight variables of the layer.