flavour
/

vtde-dinov2-small-multilingual-e5-small

Feature Extraction

vision-text-dual-encoder

Model card Files Files and versions

vtde-dinov2-small-multilingual-e5-small / tokenizer_config.json

marcusinthesky's picture

Upload processor

b609404 about 3 years ago

Raw History Blame Contribute Delete

498 Bytes

	{
	"bos_token": "<s>",
	"clean_up_tokenization_spaces": true,
	"cls_token": "<s>",
	"eos_token": "</s>",
	"mask_token": {
	"__type": "AddedToken",
	"content": "<mask>",
	"lstrip": true,
	"normalized": true,
	"rstrip": false,
	"single_word": false
	},
	"model_max_length": 512,
	"pad_token": "<pad>",
	"processor_class": "VisionTextDualEncoderProcessor",
	"sep_token": "</s>",
	"sp_model_kwargs": {},
	"tokenizer_class": "XLMRobertaTokenizer",
	"unk_token": "<unk>"
	}