Instructions to use keras/albert_extra_extra_large_en_uncased with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- KerasHub
How to use keras/albert_extra_extra_large_en_uncased with KerasHub:
import keras_hub # Load TextClassifier model text_classifier = keras_hub.models.TextClassifier.from_preset( "hf://keras/albert_extra_extra_large_en_uncased", num_classes=2, ) # Fine-tune text_classifier.fit(x=["Thilling adventure!", "Total snoozefest."], y=[1, 0]) # Classify text text_classifier.predict(["Not my cup of tea."])import keras_hub # Create a MaskedLM model task = keras_hub.models.MaskedLM.from_preset("hf://keras/albert_extra_extra_large_en_uncased")import keras_hub # Create a Backbone model unspecialized for any task backbone = keras_hub.models.Backbone.from_preset("hf://keras/albert_extra_extra_large_en_uncased") - Keras
How to use keras/albert_extra_extra_large_en_uncased with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://keras/albert_extra_extra_large_en_uncased") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from keras/albert_extra_extra_large_en_uncased: direct link, hf CLI and curl.
- Browser
- Download file 6.74 kB
-
https://huggingface.co/keras/albert_extra_extra_large_en_uncased/resolve/main/README.md
- Command line
-
hf download hf://keras/albert_extra_extra_large_en_uncased/README.md
-
curl -L -o README.md https://huggingface.co/keras/albert_extra_extra_large_en_uncased/resolve/main/README.md
6.74 kB
| library_name: keras-hub | |
| license: apache-2.0 | |
| tags: | |
| - text-classification | |
| - keras | |
| pipeline_tag: text-classification | |
| ### Model Overview | |
| ALBERT encoder network. | |
| This class implements a bi-directional Transformer-based encoder as | |
| described in | |
| ["ALBERT: A Lite BERT for Self-supervised Learning of Language Representations"](https://arxiv.org/abs/1909.11942). | |
| ALBERT is a more efficient variant of BERT, and uses parameter reduction | |
| techniques such as cross-layer parameter sharing and factorized embedding | |
| parameterization. This model class includes the embedding lookups and | |
| transformer layers, but not the masked language model or sentence order | |
| prediction heads. | |
| The default constructor gives a fully customizable, randomly initialized | |
| ALBERT encoder with any number of layers, heads, and embedding dimensions. | |
| To load preset architectures and weights, use the `from_preset` constructor. | |
| Disclaimer: Pre-trained models are provided on an "as is" basis, without | |
| warranties or conditions of any kind. | |
| ## Links | |
| * [ALBERT Quickstart Notebook](https://www.kaggle.com/code/laxmareddypatlolla/albert-quickstart-notebook) | |
| * [ALBERT API Documentation](https://keras.io/keras_hub/api/models/albert/) | |
| * [ALBERT Model Card](https://huggingface.co/docs/transformers/en/model_doc/albert) | |
| * [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/) | |
| * [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/) | |
| ## Installation | |
| Keras and KerasHub can be installed with: | |
| ``` | |
| pip install -U -q keras-hub | |
| pip install -U -q keras | |
| ``` | |
| Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page. | |
| ## Presets | |
| The following model checkpoints are provided by the Keras team. Full code examples for each are available below. | |
| | Preset name | Parameters | Description | | |
| |----------------|------------|--------------------------------------------------| | |
| | albert_base_en_uncased | 11.68M | 12-layer ALBERT model where all input is lowercased. Trained on English Wikipedia + BooksCorpus.| | |
| | albert_large_en_uncased| 17.68M | 24-layer ALBERT model where all input is lowercased. Trained on English Wikipedia + BooksCorpus. | | |
| | albert_extra_large_en_uncased | 58.72M | 24-layer ALBERT model where all input is lowercased. Trained on English Wikipedia + BooksCorpus. | | |
| | albert_extra_extra_large_en_uncased| 222.60M | 12-layer ALBERT model where all input is lowercased. Trained on English Wikipedia + BooksCorpus. | | |
| __Arguments__ | |
| - __vocabulary_size__: int. The size of the token vocabulary. | |
| - __num_layers__: int, must be divisible by `num_groups`. The number of | |
| "virtual" layers, i.e., the total number of times the input sequence | |
| will be fed through the groups in one forward pass. The input will | |
| be routed to the correct group based on the layer index. | |
| - __num_heads__: int. The number of attention heads for each transformer. | |
| The hidden size must be divisible by the number of attention heads. | |
| - __embedding_dim__: int. The size of the embeddings. | |
| - __hidden_dim__: int. The size of the transformer encoding and pooler layers. | |
| - __intermediate_dim__: int. The output dimension of the first Dense layer in | |
| a two-layer feedforward network for each transformer. | |
| - __num_groups__: int. Number of groups, with each group having | |
| `num_inner_repetitions` number of `TransformerEncoder` layers. | |
| - __num_inner_repetitions__: int. Number of `TransformerEncoder` layers per | |
| group. | |
| - __dropout__: float. Dropout probability for the Transformer encoder. | |
| - __max_sequence_length__: int. The maximum sequence length that this encoder | |
| can consume. If None, `max_sequence_length` uses the value from | |
| sequence length. This determines the variable shape for positional | |
| embeddings. | |
| - __num_segments__: int. The number of types that the 'segment_ids' input can | |
| take. | |
| ## Example Usage | |
| ```python | |
| import keras | |
| import keras_hub | |
| import numpy as np | |
| ``` | |
| Raw string data. | |
| ```python | |
| features = ["The quick brown fox jumped.", "I forgot my homework."] | |
| labels = [0, 3] | |
| # Pretrained classifier. | |
| classifier = keras_hub.models.AlbertClassifier.from_preset( | |
| "albert_extra_extra_large_en_uncased", | |
| num_classes=4, | |
| ) | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| classifier.predict(x=features, batch_size=2) | |
| # Re-compile (e.g., with a new learning rate). | |
| classifier.compile( | |
| loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True), | |
| optimizer=keras.optimizers.Adam(5e-5), | |
| jit_compile=True, | |
| ) | |
| # Access backbone programmatically (e.g., to change `trainable`). | |
| classifier.backbone.trainable = False | |
| # Fit again. | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| ``` | |
| Preprocessed integer data. | |
| ```python | |
| features = { | |
| "token_ids": np.ones(shape=(2, 12), dtype="int32"), | |
| "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2), | |
| "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2), | |
| } | |
| labels = [0, 3] | |
| # Pretrained classifier without preprocessing. | |
| classifier = keras_hub.models.AlbertClassifier.from_preset( | |
| "albert_extra_extra_large_en_uncased", | |
| num_classes=4, | |
| preprocessor=None, | |
| ) | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| ``` | |
| ## Example Usage with Hugging Face URI | |
| ```python | |
| import keras | |
| import keras_hub | |
| import numpy as np | |
| ``` | |
| Raw string data. | |
| ```python | |
| features = ["The quick brown fox jumped.", "I forgot my homework."] | |
| labels = [0, 3] | |
| # Pretrained classifier. | |
| classifier = keras_hub.models.AlbertClassifier.from_preset( | |
| "hf://keras/albert_extra_extra_large_en_uncased", | |
| num_classes=4, | |
| ) | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| classifier.predict(x=features, batch_size=2) | |
| # Re-compile (e.g., with a new learning rate). | |
| classifier.compile( | |
| loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True), | |
| optimizer=keras.optimizers.Adam(5e-5), | |
| jit_compile=True, | |
| ) | |
| # Access backbone programmatically (e.g., to change `trainable`). | |
| classifier.backbone.trainable = False | |
| # Fit again. | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| ``` | |
| Preprocessed integer data. | |
| ```python | |
| features = { | |
| "token_ids": np.ones(shape=(2, 12), dtype="int32"), | |
| "segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2), | |
| "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2), | |
| } | |
| labels = [0, 3] | |
| # Pretrained classifier without preprocessing. | |
| classifier = keras_hub.models.AlbertClassifier.from_preset( | |
| "hf://keras/albert_extra_extra_large_en_uncased", | |
| num_classes=4, | |
| preprocessor=None, | |
| ) | |
| classifier.fit(x=features, y=labels, batch_size=2) | |
| ``` | |