Spaces:

---

litert-community

/

README

like 2

Running

@jlotti This isn't Gemma 4-specific it's a transformers version incompatibility with stable litert-torch.
transformers 5.13.0 renamed the abstract method on CacheLayerMixin from get_max_cache_shape to get_max_length. Stable litert-torch 0.9.1 only implements get_max_cache_shape, and its transformers dependency is unpinned, so a fresh venv pulls 5.13+ and every cache layer subclass (including LiteRTLMCacheLayerForGemma4) ends up with an unimplemented abstract method hence the TypeError.
Two fixes:
Pin transformers: pip install "transformers<5.13" (5.12.1 works with 0.9.1)
Or use the nightly, which already implements both methods and is what the official Gemma 4 conversion docs use: pip install --pre litert-torch-nightly (https://developers.google.com/edge/litert-lm/models/gemma-4)
The docs command also passes --jinja_chat_template_override=litert-community/gemma-4-E2B-it-litert-lm (swap in the E4B repo for your model) so the bundled chat template matches the official releases. The CUDA driver warning in your log is unrelated export runs on CPU.
A pinned transformers requirement in the package would prevent this whole class of breakage.

This issue is caused by an incompatibility between the LiteRT-LM runtime's template parser and the Jinja chat template included in the Gemma 4 model on Hugging Face. Specifically, the template uses the map.get() method, a syntax that is not supported by the mobile-side parser.
Official documentation recommends using the --jinja_chat_template_override parameter to resolve this issue.

litert-torch export_hf \
--model=/media/user/1B22F52D7210D721/gemma-4-E4B-it \
--output_dir=/home/user/AI2/gemma-4-e4b-it \
--bundle_litert_lm=true \
--externalize_embedder=true \
--jinja_chat_template_override=litert-community/gemma-4-E2B-it-litert-lm

doc:https://developers.google.com/edge/litert/conversion/pytorch/genai#jinja-template-override

Google翻訳を使用

LiteRT-LMランタイムのテンプレートパーサーと、Hugging Face上のGemma 4モデルに含まれるJinjaチャットテンプレートとの間の非互換性が原因です。具体的には、そのテンプレートがモバイル側のパーサーでサポートされていない構文である map.get() メソッドを使用していることが原因です。
この問題を解決するために --jinja_chat_template_override パラメータを使用することが推奨されています。

litert-torch export_hf \
--model=/media/user/1B22F52D7210D721/gemma-4-E4B-it \
--output_dir=/home/user/AI2/gemma-4-e4b-it \
--bundle_litert_lm=true \
--externalize_embedder=true \
--jinja_chat_template_override=litert-community/gemma-4-E2B-it-litert-lm

The revious error was fixed in nightly build. I spent about 3 hours converting the model, but got an error about chat template. I have a few questions:

1. Where can I read more about litert-torch parameters? The "litert-torch export_hf --help" menu is not very expressive.
2. Can I somehow use already outputed files by litert-torch so that I don't have to recompute everything from scratch?

.
├── chat_template.jinja
├── embedder_quantized.tflite
├── embedder.tflite
├── model_quantized.tflite
├── model.tflite
├── per_layer_embedder_quantized.tflite
├── per_layer_embedder.tflite
├── tokenizer_config.json
└── tokenizer.json

3. Conversion of Gemma-4-E4B-it used about 80-85 GB of RAM. Is there a way to somehow reduce RAM consamption?
4. Which parameters were used for the official gemma-4-e4b-it.ltertlm?

litert-torch export_hf --model=/media/user/disk1/gemma-4-E4B-it/ --output_dir=/home/user/AI2/gemma-4-e4b-it/ --jinja_chat_template_override=/media/user/disk1/gemma-4-E4B-it --bundle_litert_lm=true --externalize_embedder=true --keep_temporary_files=false --quantization_recipe=dynamic_wi8_afp32

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1784413145.262674    5845 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
/home/user/AI2/venv/lib/python3.11/site-packages/torch/cuda/__init__.py:188: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 12000). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at /__w/pytorch/pytorch/c10/cuda/CUDAFunctions.cpp:119.)
  return torch._C._cuda_getDeviceCount() > 0
W0719 01:19:35.110000 5845 torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
W0719 01:20:02.187000 5845 torch/utils/_pytree.py:630] <enum 'ScaleCalculationMode'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
/home/user/AI2/venv/lib/python3.11/site-packages/torch/cuda/__init__.py:1112: UserWarning: Can't initialize NVML
  raw_cnt = _raw_device_count_nvml()
============== Export Configuration ==============
aot_backend            : None
aot_compilation_config_dict : None
aot_soc_model          : None
assistant_model        : None
auto_model_override    : None
batch_size             : 1
bundle_litert_lm       : 'true'
cache_implementation   : 'LiteRTLMCache'
cache_length           : 4096
cache_length_dim       : None
enable_dynamic_shape   : False
experimental_lightweight_conversion : False
experimental_transpile_chat_template_for_minijinja : False
experimental_use_fp16  : False
experimental_use_mixed_precision : False
export_audio_encoder   : False
export_vision_encoder  : False
externalize_embedder   : 'true'
externalize_rope       : False
extra_kwargs           : {}
input_sec              : 1.0
jinja_chat_template_override : '/media/user/disk1/gemma-4-E4B-it'
k_ts_idx               : 2
keep_temporary_files   : 'false'
litert_lm_llm_metadata_override : None
litert_lm_model_type_override : None
llm_metadata_max_num_tokens_override : None
model                  : '/media/user/disk1/gemma-4-E4B-it/'
mtp_verifier_step      : 5
output_dir             : '/home/user/AI2/gemma-4-e4b-it/'
prefill_length_dim     : None
prefill_lengths        : [128]
quantization_recipe    : 'dynamic_wi8_afp32'
sampler_temperature    : None
sampler_top_k          : None
sampler_top_p          : None
single_token_embedder  : False
split_cache            : False
stateful_after         : -1
task                   : <ExportTask.TEXT_GENERATION: 'text_generation'>
tokenizer_path_override : None
trust_remote_code      : False
use_jinja_template     : True
use_random_weights     : False
v_ts_idx               : 3
vision_encoder_quantization_recipe : 'dynamic_wi8_afp32'
work_dir               : '/home/user/AI2/gemma-4-e4b-it/'
==================================================
(00:00) [START] LiteRT GenAI Export
(00:00) [START] LiteRT GenAI Export > Load source model
Gemma4 patch applied.
Loading weights: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2076/2076 [05:02<00:00,  6.86it/s]
(05:07) [ DONE] LiteRT GenAI Export > Load source model (+05:07)
(05:07) [START] LiteRT GenAI Export > Export text prefill-decode model
Using Gemma4 exportables.
(05:07) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert
(05:07) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: prefill_128
[transformers] `get_max_cache_shape` is deprecated, and will be removed in version 5.16. Please use `get_max_length` instead
(05:23) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: prefill_128 > ExportedProgram Run Decompositions
(05:45) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: prefill_128 > ExportedProgram Run Decompositions (+00:21)
(05:45) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: prefill_128 (+00:37)
(05:45) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: decode
(05:59) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: decode > ExportedProgram Run Decompositions
(06:21) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: decode > ExportedProgram Run Decompositions (+00:22)
(06:21) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Torch Export: decode (+00:36)
(06:21) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes
(06:22) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(06:22) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(06:24) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(06:24) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(06:25) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run FX Passes (+00:03)
(06:27) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128
(06:27) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > ExportedProgram Run Decompositions
(06:45) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > ExportedProgram Run Decompositions (+00:18)
(06:45) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > ExportedProgram Run Decompositions
(06:45) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > ExportedProgram Run Decompositions (+00:00)
(06:45) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > Create MLIR Module
WARNING:jax._src.xla_bridge:An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.
(37:00) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 > Create MLIR Module (+30:15)
(37:03) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: prefill_128 (+30:36)
(37:04) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode
(37:07) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > ExportedProgram Run Decompositions
(40:36) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > ExportedProgram Run Decompositions (+03:28)
(40:36) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > ExportedProgram Run Decompositions
(40:36) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > ExportedProgram Run Decompositions (+00:00)
(40:36) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > Create MLIR Module
(41:01) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode > Create MLIR Module (+00:24)
(41:01) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Lower to MLIR: decode (+03:56)
(41:01) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Merge MLIR Modules
(41:01) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Merge MLIR Modules (+00:00)
(41:01) [START] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run LiteRT Converter Passes
(86:56) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert > Run LiteRT Converter Passes (+45:54)
(87:38) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > LiteRT-Torch Convert (+82:31)
(90:44) [START] LiteRT GenAI Export > Export text prefill-decode model > Write Model to /home/user/AI2/gemma-4-e4b-it/model.tflite
Module size is greater than 2GB
I0000 00:00:1784418659.855072    5845 flatbuffer_export.cc:4342] Estimated count of arithmetic ops: 582.296 G  ops, equivalently 291.148 G  MACs
(93:27) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > Write Model to /home/user/AI2/gemma-4-e4b-it/model.tflite (+02:42)
(93:43) [START] LiteRT GenAI Export > Export text prefill-decode model > Quantize model
Generating Quantization Parameters:: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7563/7563 [03:37<00:00, 34.84it/s]
Applying Transformations to tensors:: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 10054/10054 [00:00<00:00, 13240.73it/s]
Model name: /home/user/AI2/gemma-4-e4b-it/model.tflite
Original model size: 17.31 GiB
Quantized model size: 4.36 GiB
Quantization Ratio: 0.25 (4.0x smaller)
Total time: 221.61 s
(97:29) [ DONE] LiteRT GenAI Export > Export text prefill-decode model > Quantize model (+03:46)
(97:34) [ DONE] LiteRT GenAI Export > Export text prefill-decode model (+92:26)
(97:34) [START] LiteRT GenAI Export > Export embedder model
(97:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert
(97:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: decode_embedder
(97:42) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: decode_embedder > ExportedProgram Run Decompositions
(98:36) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: decode_embedder > ExportedProgram Run Decompositions (+00:53)
(98:36) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: decode_embedder (+00:57)
(98:36) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: prefill_embedder_128
(98:36) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: prefill_embedder_128 > ExportedProgram Run Decompositions
(98:36) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: prefill_embedder_128 > ExportedProgram Run Decompositions (+00:00)
(98:36) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Torch Export: prefill_embedder_128 (+00:00)
(98:36) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes
(98:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(98:38) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(98:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(98:38) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(98:38) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run FX Passes (+00:01)
(98:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder
(98:38) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > ExportedProgram Run Decompositions
(98:39) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > ExportedProgram Run Decompositions (+00:00)
(98:39) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > ExportedProgram Run Decompositions
(98:39) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > ExportedProgram Run Decompositions (+00:00)
(98:39) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > Create MLIR Module
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder > Create MLIR Module (+02:22)
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_embedder (+02:23)
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > ExportedProgram Run Decompositions
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > ExportedProgram Run Decompositions (+00:00)
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > ExportedProgram Run Decompositions
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > ExportedProgram Run Decompositions (+00:00)
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > Create MLIR Module
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 > Create MLIR Module (+00:00)
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_embedder_128 (+00:00)
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Merge MLIR Modules
(101:02) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Merge MLIR Modules (+00:00)
(101:02) [START] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run LiteRT Converter Passes
(101:03) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert > Run LiteRT Converter Passes (+00:00)
(101:03) [ DONE] LiteRT GenAI Export > Export embedder model > LiteRT-Torch Convert (+03:24)
(101:03) [START] LiteRT GenAI Export > Export embedder model > Write Model to /home/user/AI2/gemma-4-e4b-it/embedder.tflite
Module size is greater than 2GB
I0000 00:00:1784419277.120423    5845 flatbuffer_export.cc:4342] Estimated count of arithmetic ops: 0.330 M  ops, equivalently 0.165 M  MACs
(101:04) [ DONE] LiteRT GenAI Export > Export embedder model > Write Model to /home/user/AI2/gemma-4-e4b-it/embedder.tflite (+00:01)
(101:04) [START] LiteRT GenAI Export > Export embedder model > Quantize model
Model name: /home/user/AI2/gemma-4-e4b-it/embedder.tflite
Original model size: 2.50 GiB
Quantized model size: 646.01 MiB
Quantization Ratio: 0.25 (4.0x smaller)
Total time: 7.25 s
(101:12) [ DONE] LiteRT GenAI Export > Export embedder model > Quantize model (+00:07)
(101:14) [ DONE] LiteRT GenAI Export > Export embedder model (+03:39)
(101:14) [START] LiteRT GenAI Export > Export per_layer_embedder model
(101:14) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert
(101:14) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: decode_per_layer_embedder
(101:17) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: decode_per_layer_embedder > ExportedProgram Run Decompositions
(101:27) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: decode_per_layer_embedder > ExportedProgram Run Decompositions (+00:09)
(101:27) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: decode_per_layer_embedder (+00:12)
(101:27) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: prefill_per_layer_embedder_128
(101:29) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions
(101:37) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions (+00:07)
(101:37) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Torch Export: prefill_per_layer_embedder_128 (+00:09)
(101:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes
(101:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(101:37) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(101:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions
(101:37) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes > ExportedProgram Run Decompositions (+00:00)
(101:37) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run FX Passes (+00:00)
(101:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder
(101:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > ExportedProgram Run Decompositions
(101:45) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > ExportedProgram Run Decompositions (+00:07)
(101:45) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > ExportedProgram Run Decompositions
(101:45) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > ExportedProgram Run Decompositions (+00:00)
(101:45) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > Create MLIR Module
(135:21) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder > Create MLIR Module (+33:36)
(135:26) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: decode_per_layer_embedder (+33:49)
(135:28) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128
(135:37) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions
(140:38) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions (+05:01)
(140:38) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions
(140:38) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > ExportedProgram Run Decompositions (+00:00)
(140:38) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > Create MLIR Module
(140:43) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 > Create MLIR Module (+00:05)
(140:43) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Lower to MLIR: prefill_per_layer_embedder_128 (+05:15)
(140:43) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Merge MLIR Modules
(140:43) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Merge MLIR Modules (+00:00)
(140:43) [START] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run LiteRT Converter Passes
(140:49) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert > Run LiteRT Converter Passes (+00:05)
(140:49) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > LiteRT-Torch Convert (+39:34)
(140:49) [START] LiteRT GenAI Export > Export per_layer_embedder model > Write Model to /home/user/AI2/gemma-4-e4b-it/per_layer_embedder.tflite
Module size is greater than 2GB
I0000 00:00:1784421663.563599    5845 flatbuffer_export.cc:4342] Estimated count of arithmetic ops: 1.387 M  ops, equivalently 0.694 M  MACs
(142:54) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > Write Model to /home/user/AI2/gemma-4-e4b-it/per_layer_embedder.tflite (+02:05)
(142:54) [START] LiteRT GenAI Export > Export per_layer_embedder model > Quantize model
Model name: /home/user/AI2/gemma-4-e4b-it/per_layer_embedder.tflite
Original model size: 10.50 GiB
Quantized model size: 2.63 GiB
Quantization Ratio: 0.25 (4.0x smaller)
Total time: 674.98 s
(154:20) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model > Quantize model (+11:25)
(154:42) [ DONE] LiteRT GenAI Export > Export per_layer_embedder model (+53:28)
(154:42) [START] LiteRT GenAI Export > Export tokenizer
(156:09) [ DONE] LiteRT GenAI Export > Export tokenizer (+01:26)
(156:09) [START] LiteRT GenAI Export > Package model
(156:09) [ FAIL] LiteRT GenAI Export > Package model
(156:09) [ FAIL] LiteRT GenAI Export
Traceback (most recent call last):
  File "/home/user/AI2/venv/bin/litert-torch", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/litert_torch/cli.py", line 30, in main
    fire.Fire(CLI())
  File "/home/user/AI2/venv/lib/python3.11/site-packages/fire/core.py", line 135, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/fire/core.py", line 468, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/fire/core.py", line 684, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/litert_torch/generative/export_hf/export.py", line 211, in export
    exported_model_artifacts = run_export_tasks(
                               ^^^^^^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/3.11.15/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/litert_torch/generative/export_hf/export.py", line 68, in run_export_tasks
    exported_model_artifacts = export_task(
                               ^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/3.11.15/lib/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/AI2/venv/lib/python3.11/site-packages/litert_torch/generative/export_hf/core/litert_lm_builder.py", line 346, in package_model
    with open(chat_templates_path, 'rt') as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IsADirectoryError: [Errno 21] Is a directory: '/media/user/disk1/gemma-4-E4B-it'