{"id":3673,"date":"2025-10-23T17:43:55","date_gmt":"2025-10-23T15:43:55","guid":{"rendered":"https:\/\/ru.davidka.net\/ru\/?p=3673"},"modified":"2025-11-02T14:33:13","modified_gmt":"2025-11-02T12:33:13","slug":"%e2%9a%a1-%d0%be%d0%bf%d1%82%d0%b8%d0%bc%d0%b8%d0%b7%d0%b8%d1%80%d0%be%d0%b2%d0%b0%d0%bd%d0%bd%d1%8b%d0%b9-%d0%b2%d0%b0%d1%80%d0%b8%d0%b0%d0%bd%d1%82-%d1%81-optimum-%d0%b8-onnxruntime","status":"publish","type":"post","link":"https:\/\/ru.davidka.net\/ru\/%D0%BE%D0%BF%D1%82%D0%B8%D0%BC%D0%B8%D0%B7%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%BD%D1%8B%D0%B9-%D0%B2%D0%B0%D1%80%D0%B8%D0%B0%D0%BD%D1%82-%D1%81-optimum-%D0%B8-onnxruntime\/","title":{"rendered":"\u26a1 \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043c\u043e\u0434\u0435\u043b\u0438 \u0438\u0437 Hugging Face \u0438 \u044d\u043a\u0441\u043f\u043e\u0440\u0442 \u0432 \u0444\u043e\u0440\u043c\u0430\u0442 ONNX. \u041e\u043f\u0442\u0438\u043c\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0439 \u0432\u0430\u0440\u0438\u0430\u043d\u0442 \u0441 optimum \u0438 onnxruntime"},"content":{"rendered":"\n
from transformers import AutoTokenizer, pipeline\nfrom optimum.onnxruntime import ORTModelForSequenceClassification\nfrom optimum.onnxruntime.configuration import AutoQuantizationConfig\nfrom optimum.onnxruntime import ORTQuantizer\nfrom pathlib import Path\n\n# === CONFIGURATION ===\nmodel_id: str = "hypo69\/my_model_from_existing_datasets"\nonnx_dir: Path = Path(".\/onnx-model")\nonnx_dir.mkdir(exist_ok=True)\n\n# === STEP 1: LOAD ORIGINAL MODEL AND TOKENIZER ===\ntokenizer = AutoTokenizer.from_pretrained(model_id)\nmodel = ORTModelForSequenceClassification.from_pretrained(model_id, export=True)  # auto-export to ONNX\n\n# === STEP 2: SAVE THE EXPORTED MODEL ===\nmodel.save_pretrained(onnx_dir)\ntokenizer.save_pretrained(onnx_dir)\n\n# === STEP 3: QUANTIZATION (REDUCE SIZE, INCREASE SPEED) ===\nquantized_dir = onnx_dir \/ "quantized"\nquantized_dir.mkdir(exist_ok=True)\n\nquantizer = ORTQuantizer.from_pretrained(model)\nqconfig = AutoQuantizationConfig.avx512_vnni(is_static=False)  # choose dynamically if needed\nquantizer.quantize(save_dir=quantized_dir, quantization_config=qconfig)\n\n# === STEP 4: LOAD OPTIMIZED MODEL ===\noptimized_model = ORTModelForSequenceClassification.from_pretrained(quantized_dir)\noptimized_tokenizer = AutoTokenizer.from_pretrained(quantized_dir)\n\n# === STEP 5: CREATE PIPELINE ===\nclassifier = pipeline("text-classification", model=optimized_model, tokenizer=optimized_tokenizer)\n\n# === STEP 6: RUN INFERENCE ===\ntext = "ONNX Runtime with quantization makes inference super fast!"\nresult = classifier(text)\nprint(result)\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
\ud83d\udcca \u0427\u0442\u043e \u0434\u0435\u043b\u0430\u0435\u0442 \u044d\u0442\u043e\u0442 \u043a\u043e\u0434<\/h2>\n\n\n\n
\u042d\u0442\u0430\u043f<\/th>\u041e\u043f\u0438\u0441\u0430\u043d\u0438\u0435<\/th><\/tr><\/thead>
1. \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430<\/strong><\/td>\u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u043c\u043e\u0434\u0435\u043b\u044c \u0438\u0437 Hugging Face Hub \u0438 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u044d\u043a\u0441\u043f\u043e\u0440\u0442\u0438\u0440\u0443\u0435\u0442 \u0432 \u0444\u043e\u0440\u043c\u0430\u0442 ONNX<\/td><\/tr>
2. \u0421\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435<\/strong><\/td>\u0421\u043e\u0445\u0440\u0430\u043d\u044f\u0435\u0442 \u043c\u043e\u0434\u0435\u043b\u044c \u0438 \u0442\u043e\u043a\u0435\u043d\u0438\u0437\u0430\u0442\u043e\u0440 \u0432 \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0443\u044e \u043f\u0430\u043f\u043a\u0443<\/td><\/tr>
3. \u041a\u0432\u0430\u043d\u0442\u043e\u0432\u0430\u043d\u0438\u0435 (Quantization)<\/strong><\/td>\u041f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442 AVX512_VNNI<\/code> \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u044e (\u0441\u043e\u043a\u0440\u0430\u0449\u0430\u0435\u0442 \u0440\u0430\u0437\u043c\u0435\u0440 \u043c\u043e\u0434\u0435\u043b\u0438 \u0432 3\u20134 \u0440\u0430\u0437\u0430 \u0438 \u0443\u0441\u043a\u043e\u0440\u044f\u0435\u0442 \u0438\u043d\u0444\u0435\u0440\u0435\u043d\u0441)<\/td><\/tr>
4. \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u043c\u043e\u0434\u0435\u043b\u0438<\/strong><\/td>\u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u0443\u0436\u0435 \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u0443\u044e ONNX-\u043c\u043e\u0434\u0435\u043b\u044c<\/td><\/tr>
5. \u041f\u0430\u0439\u043f\u043b\u0430\u0439\u043d<\/strong><\/td>\u0421\u043e\u0437\u0434\u0430\u0451\u0442 transformers.pipeline<\/code> \u0441 ONNX backend<\/td><\/tr>
6. \u0418\u043d\u0444\u0435\u0440\u0435\u043d\u0441<\/strong><\/td>\u041c\u0433\u043d\u043e\u0432\u0435\u043d\u043d\u043e \u0430\u043d\u0430\u043b\u0438\u0437\u0438\u0440\u0443\u0435\u0442 \u0442\u0435\u043a\u0441\u0442 \u2014 \u0441\u043a\u043e\u0440\u043e\u0441\u0442\u044c \u0432\u044b\u0448\u0435 \u0432 2\u20135 \u0440\u0430\u0437, \u0447\u0435\u043c \u0443 PyTorch\/TensorFlow<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n
\u2699\ufe0f \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442<\/h2>\n\n\n\n
[{'label': 'POSITIVE', 'score': 0.99976}]\n<\/code><\/pre>\n\n\n\n
\n\n\n\n
\ud83d\udca1 \u0414\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043c\u043e\u0436\u043d\u043e:<\/h2>\n\n\n\n
    \n
  • \u0437\u0430\u043c\u0435\u043d\u0438\u0442\u044c AutoQuantizationConfig.avx512_vnni<\/code> \u043d\u0430 AutoQuantizationConfig.arm64()<\/code> \u2014 \u0434\u043b\u044f ARM (\u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, Mac M1\/M2);<\/li>\n\n\n\n
  • \u0434\u043e\u0431\u0430\u0432\u0438\u0442\u044c "provider": "CUDAExecutionProvider"<\/code> \u0432 from_pretrained()<\/code> \u2014 \u0434\u043b\u044f \u0443\u0441\u043a\u043e\u0440\u0435\u043d\u0438\u044f \u043d\u0430 GPU;<\/li>\n\n\n\n
  • \u0432\u043a\u043b\u044e\u0447\u0438\u0442\u044c "use_io_binding=True"<\/code> \u2014 \u0434\u043b\u044f \u0431\u0430\u0442\u0447\u0435\u0432\u043e\u0433\u043e \u0438\u043d\u0444\u0435\u0440\u0435\u043d\u0441\u0430.<\/li>\n<\/ul>\n\n\n\n
    \n\n\n\n
    <\/p>\n","protected":false},"excerpt":{"rendered":"
    \ud83d\udcca \u0427\u0442\u043e \u0434\u0435\u043b\u0430\u0435\u0442 \u044d\u0442\u043e\u0442 \u043a\u043e\u0434 \u042d\u0442\u0430\u043f \u041e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 1. \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u043c\u043e\u0434\u0435\u043b\u044c \u0438\u0437 Hugging Face Hub \u0438 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u044d\u043a\u0441\u043f\u043e\u0440\u0442\u0438\u0440\u0443\u0435\u0442 \u0432 \u0444\u043e\u0440\u043c\u0430\u0442 ONNX 2. \u0421\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0421\u043e\u0445\u0440\u0430\u043d\u044f\u0435\u0442 \u043c\u043e\u0434\u0435\u043b\u044c \u0438 \u0442\u043e\u043a\u0435\u043d\u0438\u0437\u0430\u0442\u043e\u0440 \u0432 \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u0443\u044e \u043f\u0430\u043f\u043a\u0443 3. \u041a\u0432\u0430\u043d\u0442\u043e\u0432\u0430\u043d\u0438\u0435 (Quantization) \u041f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442 AVX512_VNNI \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u044e (\u0441\u043e\u043a\u0440\u0430\u0449\u0430\u0435\u0442 \u0440\u0430\u0437\u043c\u0435\u0440 \u043c\u043e\u0434\u0435\u043b\u0438 \u0432 3\u20134 \u0440\u0430\u0437\u0430 \u0438 \u0443\u0441\u043a\u043e\u0440\u044f\u0435\u0442 \u0438\u043d\u0444\u0435\u0440\u0435\u043d\u0441) 4. \u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e\u0439 \u043c\u043e\u0434\u0435\u043b\u0438 \u0417\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u0443\u0436\u0435 \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u0443\u044e ONNX-\u043c\u043e\u0434\u0435\u043b\u044c 5. \u041f\u0430\u0439\u043f\u043b\u0430\u0439\u043d… <\/p>\n","protected":false},"author":1,"featured_media":3727,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"","neve_meta_content_width":0,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[1,965],"tags":[961,966,962,963,964,960],"class_list":["post-3673","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","category-onnx","tag-autotokenizer","tag-onnx","tag-onnxruntime","tag-optimum-onnxruntime","tag-tokenizer","tag-transformers"],"acf":[],"_links":{"self":[{"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/posts\/3673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/comments?post=3673"}],"version-history":[{"count":0,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/posts\/3673\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/media\/3727"}],"wp:attachment":[{"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/media?parent=3673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/categories?post=3673"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ru.davidka.net\/ru\/wp-json\/wp\/v2\/tags?post=3673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}