{"id":103760,"date":"2025-03-30T02:57:56","date_gmt":"2025-03-29T22:27:56","guid":{"rendered":"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/"},"modified":"2025-03-30T02:57:56","modified_gmt":"2025-03-29T22:27:56","slug":"building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6","status":"publish","type":"post","link":"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/","title":{"rendered":"\u0633\u0627\u062e\u062a \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc Litellm \u0628\u0627 `Async_pre_call_hook` \u0628\u0631\u0627\u06cc \u062d\u0627\u0644\u062a \u067e\u0631\u0648\u06a9\u0633\u06cc"},"content":{"rendered":"<div data-article-id=\"2365469\" id=\"article-body\">\n<p>\u0634\u0645\u0627 \u0627\u0632 \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u062f \u0648 \u0627\u0632 \u062a\u0648\u0627\u0646\u0627\u06cc\u06cc \u0622\u0646 \u062f\u0631 \u0645\u062a\u062d\u062f \u06a9\u0631\u062f\u0646 \u062f\u0633\u062a\u0631\u0633\u06cc \u0628\u0647 \u0645\u062f\u0644\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0632\u0628\u0627\u0646 \u0628\u0632\u0631\u06af (LLM) \u0642\u062f\u0631\u062f\u0627\u0646\u06cc \u0645\u06cc \u06a9\u0646\u06cc\u062f. \u0627\u0645\u0627 \u0627\u06af\u0631 \u0628\u0647 \u0628\u06cc\u0634\u062a\u0631 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f \u0686\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u062f\u061f \u0686\u0647 \u0645\u06cc \u0634\u0648\u062f \u0627\u06af\u0631 \u0634\u0645\u0627 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u062f \u06a9\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc \u067e\u0648\u06cc\u0627 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0628\u0648\u062f\u062c\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u06a9\u0646\u06cc\u062f \u060c \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc \u0627\u0645\u0646\u06cc\u062a\u06cc \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f <em>\u067e\u06cc\u0634 \u0627\u0632<\/em> \u0636\u0631\u0628\u0647 \u0632\u062f\u0646 \u0628\u0647 \u06cc\u06a9 \u0645\u062f\u0644 \u06af\u0631\u0627\u0646 \u0642\u06cc\u0645\u062a \u060c \u0627\u0635\u0644\u0627\u062d \u062f\u0631 \u067e\u0631\u0648\u0627\u0632 \u06cc\u0627 \u0631\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0645\u0646\u0637\u0642 \u067e\u06cc\u0686\u06cc\u062f\u0647 \u062a\u062c\u0627\u0631\u062a\u061f \u0631\u0648\u06cc\u062f\u0627\u062f\u0647\u0627\u06cc \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u062a\u0645\u0627\u0633 \u062a\u0644\u0641\u0646\u06cc <em>\u067e\u0633 \u0627\u0632<\/em> \u0622\u0646\u0647\u0627 \u0627\u062a\u0641\u0627\u0642 \u0645\u06cc \u0627\u0641\u062a\u062f \u060c \u0627\u0645\u0627 \u0645\u062f\u0627\u062e\u0644\u0647 \u0648\u0627\u0642\u0639\u06cc \u0646\u06cc\u0627\u0632 \u0628\u0647 \u062f\u0631\u06af\u06cc\u0631 \u0634\u062f\u0646 \u062f\u0627\u0631\u062f <em>\u067e\u06cc\u0634 \u0627\u0632<\/em> \u0639\u0645\u0644 \u0627\u0635\u0644\u06cc<\/p>\n<p>\u0627\u06cc\u0646 \u062c\u0627\u06cc\u06cc \u0627\u0633\u062a \u06a9\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc Litellm \u060c \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 <strong><code>async_pre_call_hook<\/code><\/strong>\u060c \u0628\u06cc\u0627 \u0628\u0627\u0632\u06cc \u0627\u06cc\u0646 \u0642\u0644\u0627\u0628 \u0645\u0633\u0644\u0645\u0627\u064b <strong>\u0642\u062f\u0631\u062a\u0645\u0646\u062f\u062a\u0631\u06cc\u0646 \u0646\u0642\u0637\u0647 \u0645\u062f\u0627\u062e\u0644\u0647<\/strong> \u062f\u0631 \u0686\u0631\u062e\u0647 \u0686\u0631\u062e\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u067e\u0631\u0648\u06a9\u0633\u06cc \u060c \u0628\u0647 \u0637\u0648\u0631 \u062e\u0627\u0635 \u0628\u0631\u0627\u06cc \u062a\u0648\u0633\u0639\u0647 \u062f\u0647\u0646\u062f\u06af\u0627\u0646 \u06a9\u0647 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0631\u0647\u06af\u06cc\u0631\u06cc \u060c \u062a\u062c\u0632\u06cc\u0647 \u0648 \u062a\u062d\u0644\u06cc\u0644 \u060c \u0627\u0635\u0644\u0627\u062d \u06cc\u0627 \u062d\u062a\u06cc \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc \u0645\u0633\u062f\u0648\u062f \u062f\u0627\u0631\u0646\u062f \u060c \u0637\u0631\u0627\u062d\u06cc \u0634\u062f\u0647 \u0627\u0633\u062a <strong>\u0641\u0642\u0637 \u0644\u062d\u0638\u0627\u062a\u06cc \u0642\u0628\u0644 \u0627\u0632 \u0627\u0631\u0633\u0627\u0644 \u0622\u0646\u0647\u0627 \u0628\u0647 \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u0647 Target LLM.<\/strong><\/p>\n<p>\u0627\u06cc\u0646 \u0631\u0627\u0647\u0646\u0645\u0627\u06cc \u062c\u0627\u0645\u0639 \u0634\u0645\u0627 \u0631\u0627 \u0627\u0632 \u0637\u0631\u06cc\u0642 \u0647\u0631 \u0622\u0646\u0686\u0647 \u0631\u0627 \u06a9\u0647 \u0628\u0627\u06cc\u062f \u0628\u062f\u0627\u0646\u06cc\u062f \u0628\u0631\u0627\u06cc \u062a\u0633\u0644\u0637 \u0628\u0631 \u0627\u06cc\u0646 \u0627\u0633\u062a \u060c \u0637\u06cc \u0645\u06cc \u06a9\u0646\u062f <code>async_pre_call_hook<\/code> \u0648 \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u067e\u06cc\u0634\u0631\u0641\u062a\u0647 \u0648 \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u062e\u0648\u062f \u0628\u0633\u0627\u0632\u06cc\u062f.<\/p>\n<p><strong>\u0627\u06cc\u0646 \u0631\u0627\u0647\u0646\u0645\u0627 \u0628\u0631\u0627\u06cc \u06a9\u06cc\u0633\u062a\u061f<\/strong><\/p>\n<ul>\n<li>    \u062a\u0648\u0633\u0639\u0647 \u062f\u0647\u0646\u062f\u06af\u0627\u0646 \u067e\u0627\u06cc\u062a\u0648\u0646 \u0631\u0627\u0647 \u062d\u0644 \u0647\u0627\u06cc \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u062f\u0631 \u0628\u0627\u0644\u0627\u06cc \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0627\u06cc\u062c\u0627\u062f \u0645\u06cc \u06a9\u0646\u0646\u062f.<\/li>\n<li>    \u0645\u0647\u0646\u062f\u0633\u0627\u0646 \u067e\u0644\u062a\u0641\u0631\u0645 \u06a9\u0647 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0627\u062c\u0631\u0627\u06cc \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc \u062e\u0627\u0635 \u060c \u0627\u0642\u062f\u0627\u0645\u0627\u062a \u0627\u0645\u0646\u06cc\u062a\u06cc \u06cc\u0627 \u06a9\u0646\u062a\u0631\u0644 \u0647\u0632\u06cc\u0646\u0647 \u062f\u0627\u0631\u0646\u062f.<\/li>\n<li>    \u0647\u0631\u06a9\u0633\u06cc \u06a9\u0647 \u0645\u0627\u06cc\u0644 \u0628\u0647 \u0627\u0636\u0627\u0641\u0647 \u06a9\u0631\u062f\u0646 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u060c \u0627\u0639\u062a\u0628\u0627\u0631 \u0633\u0646\u062c\u06cc \u06cc\u0627 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u063a\u0646\u06cc \u0633\u0627\u0632\u06cc \u0633\u0641\u0627\u0631\u0634\u06cc \u0628\u0647 \u0637\u0648\u0631 \u0645\u0633\u062a\u0642\u06cc\u0645 \u062f\u0631 \u0644\u0627\u06cc\u0647 \u067e\u0631\u0648\u06a9\u0633\u06cc \u0627\u0633\u062a.<\/li>\n<li>    \u062a\u0648\u0633\u0639\u0647 \u062f\u0647\u0646\u062f\u06af\u0627\u0646 \u0628\u0627 \u0647\u062f\u0641 \u0627\u06cc\u062c\u0627\u062f \u0645\u0627\u0698\u0648\u0644 \u0647\u0627\u06cc \u067e\u0644\u0627\u06af\u06cc\u0646 Litellm Sharable.<\/li>\n<\/ul>\n<p><strong>\u0622\u0646\u0686\u0647 \u06cc\u0627\u062f \u062e\u0648\u0627\u0647\u06cc\u062f \u06af\u0631\u0641\u062a:<\/strong><\/p>\n<ul>\n<li>    \u0646\u0642\u0634 \u0648 \u0632\u0645\u0627\u0646 \u062f\u0642\u06cc\u0642 \u0627\u0632 <code>async_pre_call_hook<\/code> \u062f\u0631 \u062c\u0631\u06cc\u0627\u0646 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u067e\u0631\u0648\u06a9\u0633\u06cc.<\/li>\n<li>    \u06cc\u06a9 \u0634\u06cc\u0631\u062c\u0647 \u0639\u0645\u06cc\u0642 \u0628\u0647 \u0627\u0645\u0636\u0627\u06cc \u0647\u0648\u06a9: \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u0648\u0631\u0648\u062f\u06cc \u0622\u0646 (<code>user_api_key_dict<\/code>\u0628\u0627 <code>cache<\/code>\u0628\u0627 <code>data<\/code>\u0628\u0627 <code>call_type<\/code>) \u0648 \u0627\u0647\u0645\u06cc\u062a \u0622\u0646\u0647\u0627.<\/li>\n<li>    \u0686\u06af\u0648\u0646\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646 \u0627\u0632 \u0632\u0645\u06cc\u0646\u0647 \u063a\u0646\u06cc \u0627\u0631\u0627\u0626\u0647 \u0634\u062f\u0647 \u062a\u0648\u0633\u0637 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f <code>UserAPIKeyAuth<\/code> \u0628\u0631\u0627\u06cc \u0645\u0646\u0637\u0642 \u0627\u0641\u0632\u0648\u0646\u0647 \u0622\u06af\u0627\u0647 \u06a9\u0627\u0631\u0628\u0631.<\/li>\n<li>    \u062a\u06a9\u0646\u06cc\u06a9 \u0647\u0627\u06cc \u0645\u0624\u062b\u0631 \u0628\u0631\u0627\u06cc \u0628\u0627\u0632\u0631\u0633\u06cc \u0648 \u0628\u0627 \u062e\u06cc\u0627\u0644 \u0631\u0627\u062d\u062a \u0627\u0635\u0644\u0627\u062d\u0627\u062a \u0645\u0647\u0645 <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a<\/li>\n<li>    \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 <code>DualCache<\/code> \u0628\u0631\u0627\u06cc \u0633\u0627\u062e\u062a \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u062d\u0627\u0644\u062a (\u0645\u0627\u0646\u0646\u062f \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0647\u0627\u06cc \u0646\u0631\u062e \u06cc\u0627 \u0631\u062f\u06cc\u0627\u0628 \u0647\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647).<\/li>\n<li>    \u062a\u0633\u0644\u0637 \u0628\u0631 \u0645\u0642\u0627\u062f\u06cc\u0631 \u0628\u0627\u0632\u06af\u0634\u062a \u0647\u0648\u06a9 (<code>None<\/code>\u0628\u0627 <code>dict<\/code>\u0628\u0627 <code>str<\/code>) \u0648 \u0631\u0633\u06cc\u062f\u06af\u06cc \u0628\u0647 \u0627\u0633\u062a\u062b\u0646\u0627 (<code>HTTPException<\/code>) \u0628\u0631\u0627\u06cc \u06a9\u0646\u062a\u0631\u0644 \u0633\u0631\u0646\u0648\u0634\u062a \u062f\u0631\u062e\u0648\u0627\u0633\u062a.<\/li>\n<li>    \u062f\u0633\u062a\u0648\u0631\u0627\u0644\u0639\u0645\u0644 \u0647\u0627\u06cc \u06af\u0627\u0645 \u0628\u0647 \u06af\u0627\u0645 \u0628\u0631\u0627\u06cc \u0627\u06cc\u062c\u0627\u062f \u060c \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0648 \u0622\u0632\u0645\u0627\u06cc\u0634 \u0627\u0648\u0644\u06cc\u0646 \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f.<\/li>\n<li>    \u0627\u0644\u06af\u0648\u0647\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0645\u0634\u062a\u0631\u06a9 \u0648 \u067e\u06cc\u0634\u0631\u0641\u062a\u0647: \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u067e\u0648\u06cc\u0627 \u060c \u0627\u0639\u062a\u0628\u0627\u0631 \u0633\u0646\u062c\u06cc \u0648\u0631\u0648\u062f\u06cc \u060c \u0645\u0627\u0633\u06a9 PII \u060c \u0627\u062c\u0631\u0627\u06cc \u0633\u06cc\u0627\u0633\u062a \u060c \u0631\u062f \u0633\u0641\u0627\u0631\u0634\u06cc \u060c \u062a\u0646\u0638\u06cc\u0645 \u0622\u0632\u0645\u0627\u06cc\u0634 A\/B \u0648 \u0645\u0648\u0627\u0631\u062f \u062f\u06cc\u06af\u0631.<\/li>\n<li>    \u0628\u0647\u062a\u0631\u06cc\u0646 \u0631\u0648\u0634\u0647\u0627 \u0628\u0631\u0627\u06cc \u0646\u0648\u0634\u062a\u0646 \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u0627\u062c\u0631\u0627 \u060c \u0627\u06cc\u0645\u0646 \u0648 \u0642\u0627\u0628\u0644 \u0646\u06af\u0647\u062f\u0627\u0631\u06cc.<\/li>\n<li>    \u0627\u0633\u062a\u0631\u0627\u062a\u0698\u06cc \u0647\u0627\u06cc \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0628\u0631\u0627\u06cc \u0639\u06cc\u0628 \u06cc\u0627\u0628\u06cc \u062f\u0631 \u0627\u062c\u0631\u0627\u06cc \u0642\u0644\u0627\u0628 \u062e\u0648\u062f.<\/li>\n<\/ul>\n<hr\/>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter-rtl ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">\u0641\u0647\u0631\u0633\u062a \u0645\u0637\u0627\u0644\u0628<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_1_%DA%86%D8%B1%D8%A7_async_pre_call_hook_%D8%A2%DB%8C%D8%A7_%D8%B4%D9%85%D8%A7_%D8%A8%D8%B1%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D9%87%D8%A7_%D9%87%D8%B3%D8%AA%DB%8C%D8%AF\" >\u0642\u0633\u0645\u062a 1: \u0686\u0631\u0627 async_pre_call_hook \u0622\u06cc\u0627 \u0634\u0645\u0627 \u0628\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627 \u0647\u0633\u062a\u06cc\u062f<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_2_%D8%A2%D9%86%D8%A7%D8%AA%D9%88%D9%85%DB%8C_async_pre_call_hook_%D8%A7%D9%85%D8%B6%D8%A7%D8%A1\" >\u0642\u0633\u0645\u062a 2: \u0622\u0646\u0627\u062a\u0648\u0645\u06cc async_pre_call_hook \u0627\u0645\u0636\u0627\u0621<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_3_%D8%A7%D8%AC%D8%B1%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D8%AE%D9%88%D8%AF-%DA%AF%D8%A7%D9%85_%D8%A8%D9%87_%DA%AF%D8%A7%D9%85\" >\u0642\u0633\u0645\u062a 3: \u0627\u062c\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f-\u06af\u0627\u0645 \u0628\u0647 \u06af\u0627\u0645<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_4_%D8%A7%D9%84%DA%AF%D9%88%D9%87%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D9%85%D8%B4%D8%AA%D8%B1%DA%A9_%D9%88_%D9%85%D9%88%D8%A7%D8%B1%D8%AF_%D8%A7%D8%B3%D8%AA%D9%81%D8%A7%D8%AF%D9%87\" >\u0642\u0633\u0645\u062a 4: \u0627\u0644\u06af\u0648\u0647\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0645\u0634\u062a\u0631\u06a9 \u0648 \u0645\u0648\u0627\u0631\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_5_%D8%AA%D8%B9%D8%A7%D9%85%D9%84_%D8%A8%D8%A7_%D9%82%D9%84%D8%A7%D8%A8_%D9%87%D8%A7%DB%8C_%D8%AF%DB%8C%DA%AF%D8%B1_%D9%87%D9%85%D8%A7%D9%87%D9%86%DA%AF%DB%8C_%DA%A9%D9%84%DB%8C%D8%AF%DB%8C_%D8%A7%D8%B3%D8%AA\" >\u0642\u0633\u0645\u062a 5: \u062a\u0639\u0627\u0645\u0644 \u0628\u0627 \u0642\u0644\u0627\u0628 \u0647\u0627\u06cc \u062f\u06cc\u06af\u0631 (\u0647\u0645\u0627\u0647\u0646\u06af\u06cc \u06a9\u0644\u06cc\u062f\u06cc \u0627\u0633\u062a)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_6_%D8%AA%DA%A9%D9%86%DB%8C%DA%A9_%D9%87%D8%A7%DB%8C_%D9%BE%DB%8C%D8%B4%D8%B1%D9%81%D8%AA%D9%87\" >\u0642\u0633\u0645\u062a 6: \u062a\u06a9\u0646\u06cc\u06a9 \u0647\u0627\u06cc \u067e\u06cc\u0634\u0631\u0641\u062a\u0647<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_7_%D8%A8%D9%87%D8%AA%D8%B1%DB%8C%D9%86_%D8%B4%DB%8C%D9%88%D9%87_%D9%87%D8%A7_%D9%88_%D9%85%D8%B4%DA%A9%D9%84%D8%A7%D8%AA\" >\u0642\u0633\u0645\u062a 7: \u0628\u0647\u062a\u0631\u06cc\u0646 \u0634\u06cc\u0648\u0647 \u0647\u0627 \u0648 \u0645\u0634\u06a9\u0644\u0627\u062a<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%82%D8%B3%D9%85%D8%AA_8_%D8%A7%D8%B4%DA%A9%D8%A7%D9%84_%D8%B2%D8%AF%D8%A7%DB%8C%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D8%AE%D9%88%D8%AF\" >\u0642\u0633\u0645\u062a 8: \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/nabfollower.com\/blog\/building-litellm-plugins-with-asyncprecallhook-for-proxy-mode-3lp6\/#%D9%86%D8%AA%DB%8C%D8%AC%D9%87_%DA%AF%DB%8C%D8%B1%DB%8C_%D8%B3%D8%A7%D8%AE%D8%AA_%D8%8C_%DA%AF%D8%B3%D8%AA%D8%B1%D8%B4_%D8%8C_%DA%A9%D9%86%D8%AA%D8%B1%D9%84\" >\u0646\u062a\u06cc\u062c\u0647 \u06af\u06cc\u0631\u06cc: \u0633\u0627\u062e\u062a \u060c \u06af\u0633\u062a\u0631\u0634 \u060c \u06a9\u0646\u062a\u0631\u0644<\/a><\/li><\/ul><\/nav><\/div>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_1_%DA%86%D8%B1%D8%A7_async_pre_call_hook_%D8%A2%DB%8C%D8%A7_%D8%B4%D9%85%D8%A7_%D8%A8%D8%B1%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D9%87%D8%A7_%D9%87%D8%B3%D8%AA%DB%8C%D8%AF\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 1: \u0686\u0631\u0627 <code>async_pre_call_hook<\/code> \u0622\u06cc\u0627 \u0634\u0645\u0627 \u0628\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627 \u0647\u0633\u062a\u06cc\u062f<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Litellm \u0646\u0642\u0627\u0637 \u0645\u062e\u062a\u0644\u0641 \u067e\u0627\u0633\u062e \u0628\u0647 \u062a\u0645\u0627\u0633 \u0631\u0627 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f (<code>log_success_event<\/code>\u0628\u0627 <code>log_failure_event<\/code>\u060c \u0648 \u063a\u06cc\u0631\u0647) \u060c \u062f\u0631 \u062f\u0631\u062c\u0647 \u0627\u0648\u0644 \u0628\u0631\u0627\u06cc <em>\u0631\u0639\u0627\u06cc\u062a<\/em> &#8211; \u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 \u0622\u0646\u0686\u0647 \u0627\u062a\u0641\u0627\u0642 \u0627\u0641\u062a\u0627\u062f\u0647 \u0627\u0633\u062a <em>\u067e\u0633 \u0627\u0632<\/em> \u0648\u0627\u0642\u0639\u06cc\u062a \u062f\u0631 \u062d\u0627\u0644\u06cc \u06a9\u0647 \u0636\u0631\u0648\u0631\u06cc \u0627\u0633\u062a \u060c \u0622\u0646\u0647\u0627 \u0628\u0647 \u0634\u0645\u0627 \u0627\u062c\u0627\u0632\u0647 \u0646\u0645\u06cc \u062f\u0647\u0646\u062f <em>\u062a\u063a\u06cc\u06cc\u0631<\/em> \u062f\u0648\u0631\u0647 \u0639\u0645\u0644<\/p>\n<p>\u062f\u0631 <code>async_pre_call_hook<\/code> \u0628\u0647 \u062f\u0644\u06cc\u0644 \u0648\u06cc\u0698\u06af\u06cc \u0647\u0627\u06cc \u0645\u0646\u062d\u0635\u0631 \u0628\u0647 \u0641\u0631\u062f \u0645\u062a\u0646\u0627\u0633\u0628 \u0628\u0627 \u062a\u0648\u0633\u0639\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u060c \u0627\u0632 \u0647\u0645 \u062c\u062f\u0627 \u0645\u06cc \u0634\u0648\u062f:<\/p>\n<ol>\n<li> <strong>\u0632\u0645\u0627\u0646 \u0628\u0646\u062f\u06cc \u0647\u0645\u0647 \u0686\u06cc\u0632 \u0627\u0633\u062a:<\/strong> \u0627\u06cc\u0646 \u062f\u0631 \u0628\u062d\u0631\u0627\u0646\u06cc \u062a\u0631\u06cc\u0646 \u0645\u0642\u0637\u0639 \u0627\u062c\u0631\u0627 \u0645\u06cc \u0634\u0648\u062f: <strong>\u067e\u0633 \u0627\u0632<\/strong> \u0627\u062d\u0631\u0627\u0632 \u0647\u0648\u06cc\u062a \u0627\u0648\u0644\u06cc\u0647 \u0648 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0622\u0645\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u060c \u0627\u0645\u0627 <strong>\u067e\u06cc\u0634 \u0627\u0632<\/strong> \u0641\u0631\u0627\u062e\u0648\u0627\u0646 \u0628\u0627\u0644\u0642\u0648\u0647 \u067e\u0631\u0647\u0632\u06cc\u0646\u0647 \u0648 \u0648\u0642\u062a \u06af\u06cc\u0631 \u0628\u0647 API \u0648\u0627\u0642\u0639\u06cc LLM (\u0645\u0627\u0646\u0646\u062f Openai \u060c Anthropic \u060c Cohere \u0648 \u063a\u06cc\u0631\u0647). \u0627\u06cc\u0646 \u067e\u0646\u062c\u0631\u0647 \u0637\u0644\u0627\u06cc\u06cc \u0634\u0645\u0627 \u0628\u0631\u0627\u06cc \u0645\u062f\u0627\u062e\u0644\u0647 \u0627\u0633\u062a.<\/li>\n<li> <strong>\u0645\u062f\u0627\u062e\u0644\u0647 \u060c \u0646\u0647 \u0641\u0642\u0637 \u0645\u0634\u0627\u0647\u062f\u0647:<\/strong> \u0628\u0631\u062e\u0644\u0627\u0641 \u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 \u062a\u0645\u0627\u0633 \u060c <code>async_pre_call_hook<\/code> \u0628\u0631\u0627\u06cc <strong>\u0639\u0645\u0644<\/strong>\u0628\u0634\u0631 \u0645\u0642\u062f\u0627\u0631 \u0628\u0627\u0632\u06af\u0634\u062a \u0622\u0646 \u06cc\u0627 \u0627\u0633\u062a\u062b\u0646\u0627\u0626\u0627\u062a\u06cc \u06a9\u0647 \u0627\u0641\u0632\u0627\u06cc\u0634 \u0645\u06cc \u062f\u0647\u062f \u0645\u0633\u062a\u0642\u06cc\u0645\u0627\u064b \u062f\u06cc\u06a9\u062a\u0647 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0622\u06cc\u0627 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u067e\u06cc\u0634 \u0645\u06cc \u0631\u0648\u062f \u060c \u0627\u0635\u0644\u0627\u062d \u0645\u06cc \u0634\u0648\u062f \u06cc\u0627 \u0628\u0647 \u0637\u0648\u0631 \u06a9\u0627\u0645\u0644 \u0631\u062f \u0645\u06cc \u0634\u0648\u062f.<\/li>\n<li> <strong>\u0645\u062a\u0646 \u063a\u0646\u06cc:<\/strong> \u0627\u06cc\u0646 \u0646\u0647 \u062a\u0646\u0647\u0627 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0627\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0644\u06a9\u0647 \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0645\u062a\u0646\u06cc \u062d\u06cc\u0627\u062a\u06cc \u0631\u0627 \u0646\u06cc\u0632 \u062f\u0631\u06cc\u0627\u0641\u062a \u0645\u06cc \u06a9\u0646\u062f:\n<ul>\n<li>    \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0627\u062d\u0631\u0627\u0632 \u0647\u0648\u06cc\u062a \u06a9\u0627\u0631\u0628\u0631\/\u06a9\u0644\u06cc\u062f \u062f\u0642\u06cc\u0642 (<code>UserAPIKeyAuth<\/code>).<\/li>\n<li>    \u062f\u0633\u062a\u0631\u0633\u06cc \u0628\u0647 \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u0645\u0634\u062a\u0631\u06a9 \u067e\u0631\u0648\u06a9\u0633\u06cc (<code>DualCache<\/code>).<\/li>\n<li>    \u0646\u0648\u0639 \u062e\u0627\u0635\u06cc \u0627\u0632 \u062a\u0645\u0627\u0633 \u0635\u0648\u0631\u062a \u06af\u0631\u0641\u062a\u0647 (<code>call_type<\/code>).<\/li>\n<\/ul>\n<\/li>\n<li> <strong>\u062c\u0647\u0634 \u067e\u0630\u06cc\u0631\u06cc:<\/strong> \u062f\u0631\u06cc\u0627\u0641\u062a \u0645\u06cc \u06a9\u0646\u062f <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a (\u0628\u0627\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0622\u0645\u0627\u062f\u0647 \u0634\u062f\u0647 \u0628\u0631\u0627\u06cc LLM) \u0648 \u0645\u06cc \u062a\u0648\u0627\u0646\u062f <em>\u0628\u0647 \u0637\u0648\u0631 \u0645\u0633\u062a\u0642\u06cc\u0645 \u0627\u0635\u0644\u0627\u062d \u06a9\u0646\u06cc\u062f<\/em> \u0642\u0628\u0644 \u0627\u0632 \u0627\u062c\u0627\u0632\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631\u062e\u0648\u0627\u0633\u062a.<\/li>\n<li> <strong>\u0642\u062f\u0631\u062a \u062e\u0627\u0635 \u067e\u0631\u0648\u06a9\u0633\u06cc:<\/strong> \u0627\u06cc\u0646 \u0642\u0644\u0627\u0628 \u06cc\u06a9\u06cc \u0627\u0632 \u0648\u06cc\u0698\u06af\u06cc \u0647\u0627\u06cc <strong>\u067e\u0631\u0648\u06a9\u0633\u06cc<\/strong> \u0645\u062d\u06cc\u0637 \u0632\u06cc\u0633\u062a \u060c \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0632\u06cc\u0631\u0633\u0627\u062e\u062a \u0647\u0627\u06cc \u067e\u0631\u0648\u06a9\u0633\u06cc (\u0627\u062d\u0631\u0627\u0632 \u0647\u0648\u06cc\u062a \u060c \u062d\u0627\u0641\u0638\u0647 \u067e\u0646\u0647\u0627\u0646 \u060c \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc) \u0628\u0631\u0627\u06cc \u0627\u06cc\u062c\u0627\u062f \u0645\u0646\u0637\u0642 \u067e\u06cc\u0686\u06cc\u062f\u0647 \u0627\u06cc \u06a9\u0647 \u062f\u0631 \u06cc\u06a9 \u0627\u062f\u063a\u0627\u0645 \u0633\u0627\u062f\u0647 \u06a9\u062a\u0627\u0628\u062e\u0627\u0646\u0647 \u0627\u0645\u06a9\u0627\u0646 \u067e\u0630\u06cc\u0631 \u0646\u06cc\u0633\u062a.<\/li>\n<\/ol>\n<p><strong>\u0628\u0647 \u0637\u0648\u0631 \u062e\u0644\u0627\u0635\u0647 \u060c \u0627\u06af\u0631 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u062f:<\/strong><\/p>\n<ul>\n<li>    \u0645\u062f\u0644 \u0647\u062f\u0641 \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u067e\u0648\u06cc\u0627 \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0647\u06cc\u062f.<\/li>\n<li>    \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0631\u0627 \u0627\u0636\u0627\u0641\u0647 \u060c \u062d\u0630\u0641 \u06cc\u0627 \u0627\u0635\u0644\u0627\u062d \u06a9\u0646\u06cc\u062f (<code>temperature<\/code>\u0628\u0627 <code>max_tokens<\/code>\u0628\u0627 <code>messages<\/code>\u060c \u0648 \u063a\u06cc\u0631\u0647).<\/li>\n<li>    \u0642\u0628\u0644 \u0627\u0632 \u0647\u0632\u06cc\u0646\u0647 \u06a9\u0631\u062f\u0646 \u062f\u0631 \u06cc\u06a9 \u062a\u0645\u0627\u0633 LLM \u060c \u062f\u0631 \u0628\u0631\u0627\u0628\u0631 \u0642\u0648\u0627\u0646\u06cc\u0646 \u0633\u0641\u0627\u0631\u0634\u06cc \u062a\u0623\u06cc\u06cc\u062f \u06a9\u0646\u06cc\u062f.<\/li>\n<li>    \u06a9\u0646\u062a\u0631\u0644 \u062f\u0633\u062a\u0631\u0633\u06cc \u0631\u06cc\u0632 \u062f\u0627\u0646\u0647 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0646\u0642\u0634 \u0647\u0627\u06cc \u06a9\u0627\u0631\u0628\u0631 \u060c \u0628\u0648\u062f\u062c\u0647 \u06cc\u0627 \u0645\u062c\u0648\u0632\u0647\u0627\u06cc \u0630\u062e\u06cc\u0631\u0647 \u0634\u062f\u0647 \u062f\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u06a9\u0644\u06cc\u062f\u06cc \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>    \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u062a\u062c\u0632\u06cc\u0647 \u0648 \u062a\u062d\u0644\u06cc\u0644 \u0645\u062d\u062a\u0648\u0627 \u06cc\u0627 \u0628\u0631\u0631\u0633\u06cc \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u0631\u062f \u06a9\u0646\u06cc\u062f.<\/li>\n<li>    \u06a9\u0646\u062a\u0631\u0644 \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0646\u0631\u062e \u067e\u06cc\u0634\u0631\u0641\u062a\u0647 \u06cc\u0627 \u06a9\u0646\u062a\u0631\u0644 \u0647\u0645\u0632\u0645\u0627\u0646 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>    \u0642\u0628\u0644 \u0627\u0632 \u062a\u0631\u06a9 \u0632\u06cc\u0631\u0633\u0627\u062e\u062a \u0647\u0627\u06cc \u0634\u0645\u0627 \u060c \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u062d\u0633\u0627\u0633 \u0631\u0627 \u0645\u0627\u0633\u06a9 \u06a9\u0646\u06cc\u062f.<\/li>\n<\/ul>\n<p><strong>&#8230; \u067e\u0633 <code>async_pre_call_hook<\/code> \u0642\u0644\u0627\u0628 \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632 \u0634\u0645\u0627\u0633\u062a<\/strong><\/p>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_2_%D8%A2%D9%86%D8%A7%D8%AA%D9%88%D9%85%DB%8C_async_pre_call_hook_%D8%A7%D9%85%D8%B6%D8%A7%D8%A1\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 2: \u0622\u0646\u0627\u062a\u0648\u0645\u06cc <code>async_pre_call_hook<\/code> \u0627\u0645\u0636\u0627\u0621<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u062f\u0631\u06a9 \u0648\u0631\u0648\u062f\u06cc \u0647\u0627 \u0648 \u062e\u0631\u0648\u062c\u06cc \u0647\u0627\u06cc \u0642\u0644\u0627\u0628 \u0628\u0631\u0627\u06cc \u0633\u0627\u062e\u062a \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627 \u0627\u0633\u0627\u0633\u06cc \u0627\u0633\u062a. \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0627\u0645\u0636\u0627\u06cc \u0627\u0631\u0627\u0626\u0647 \u0634\u062f\u0647 \u062a\u0648\u0633\u0637 <code>CustomLogger<\/code> \u06a9\u0644\u0627\u0633 \u067e\u0627\u06cc\u0647:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"c1\"># Defined in litellm.integrations.custom_logger.CustomLogger\n<\/span><span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span>\n    <span class=\"n\">self<\/span><span class=\"p\">,<\/span>\n    <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">:<\/span> <span class=\"n\">UserAPIKeyAuth<\/span><span class=\"p\">,<\/span>  <span class=\"c1\"># &lt;&lt;&lt; Your window into user\/key context\n<\/span>    <span class=\"n\">cache<\/span><span class=\"p\">:<\/span> <span class=\"n\">DualCache<\/span><span class=\"p\">,<\/span>                   <span class=\"c1\"># &lt;&lt;&lt; Your tool for managing state\n<\/span>    <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span>                         <span class=\"c1\"># &lt;&lt;&lt; The request payload (mutable!)\n<\/span>    <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[<\/span>                 <span class=\"c1\"># &lt;&lt;&lt; The type of LLM operation\n<\/span>        <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">text_completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">embeddings<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"sh\">\"<\/span><span class=\"s\">image_generation<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">moderation<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">audio_transcription<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"sh\">\"<\/span><span class=\"s\">pass_through_endpoint<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">rerank<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"p\">...<\/span> <span class=\"c1\"># (Potentially more types)\n<\/span>    <span class=\"p\">]<\/span>\n<span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"n\">Optional<\/span><span class=\"p\">[<\/span><span class=\"n\">Union<\/span><span class=\"p\">[<\/span><span class=\"nb\">Exception<\/span><span class=\"p\">,<\/span> <span class=\"nb\">str<\/span><span class=\"p\">,<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">]]:<\/span> <span class=\"c1\"># &lt;&lt;&lt; Determines request outcome\n<\/span>    <span class=\"c1\"># Your plugin logic goes here\n<\/span>    <span class=\"k\">pass<\/span>\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0647\u0631 \u0645\u0624\u0644\u0641\u0647 \u0631\u0627 \u062a\u062c\u0632\u06cc\u0647 \u06a9\u0646\u06cc\u0645:<\/p>\n<ul>\n<li>\n<p><strong><code>self<\/code><\/strong>: \u0646\u0645\u0648\u0646\u0647 \u06a9\u0644\u0627\u0633 \u06a9\u0646\u062a\u0631\u0644 \u06a9\u0646\u0646\u062f\u0647 \u0633\u0641\u0627\u0631\u0634\u06cc \u0634\u0645\u0627 (\u06cc\u06a9\u06cc \u0627\u0632 \u0627\u0631\u062b \u0645\u06cc \u0628\u0631\u062f <code>CustomLogger<\/code>). \u0628\u0647 \u0634\u0645\u0627 \u0627\u0645\u06a9\u0627\u0646 \u0645\u06cc \u062f\u0647\u062f \u0628\u0647 \u0633\u0627\u06cc\u0631 \u0631\u0648\u0634\u0647\u0627 \u06cc\u0627 \u0648\u06cc\u0698\u06af\u06cc \u0647\u0627\u06cc\u06cc \u06a9\u0647 \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u062f\u0631 \u06a9\u0644\u0627\u0633 \u062e\u0648\u062f \u062a\u0639\u0631\u06cc\u0641 \u06a9\u0646\u06cc\u062f \u062f\u0633\u062a\u0631\u0633\u06cc \u067e\u06cc\u062f\u0627 \u06a9\u0646\u06cc\u062f (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u0634\u062f\u0647 \u062f\u0631 \u0637\u0648\u0644 <code>__init__<\/code>).<\/p>\n<\/li>\n<li>\n<p><strong><code>user_api_key_dict: UserAPIKeyAuth<\/code><\/strong>: \u0627\u06cc\u0646 \u0645\u0633\u0644\u0645\u0627\u064b \u0628\u0627 \u0627\u0631\u0632\u0634 \u062a\u0631\u06cc\u0646 \u067e\u0627\u0631\u0627\u0645\u062a\u0631 \u0628\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u0645\u062a\u0646\u06cc \u0627\u0633\u062a. \u0627\u06cc\u0646 \u0627\u0633\u062a <strong>\u0646\u0647<\/strong> \u0641\u0642\u0637 \u06cc\u06a9 \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0633\u0627\u062f\u0647 \u060c \u0627\u0645\u0627 \u0627\u0644\u0641 <code>TypedDict<\/code> (\u062a\u0639\u0631\u06cc\u0641 \u0634\u062f\u0647 \u062f\u0631 <code>proxy\/_types.py<\/code>) \u062f\u0631 \u0645\u0631\u062d\u0644\u0647 \u0627\u062d\u0631\u0627\u0632 \u0647\u0648\u06cc\u062a \u062c\u0645\u0639\u06cc\u062a (<code>user_api_key_auth.py<\/code>). \u0627\u06cc\u0646 \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0632\u06cc\u0627\u062f\u06cc \u062f\u0631 \u0645\u0648\u0631\u062f \u06a9\u0644\u06cc\u062f API \u0645\u0639\u062a\u0628\u0631 \u0648 \u0646\u0647\u0627\u062f\u0647\u0627\u06cc \u0645\u0631\u062a\u0628\u0637 \u0628\u0627 \u0622\u0646 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f. \u0634\u0645\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0628\u0627 \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u0627\u0632 \u0632\u0645\u06cc\u0646\u0647 \u0647\u0627\u06cc\u06cc \u0645\u0627\u0646\u0646\u062f:<\/p>\n<ul>\n<li>  <code>token<\/code>: <strong>\u0647\u0634<\/strong> \u0646\u0633\u062e\u0647 \u06a9\u0644\u06cc\u062f API \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a.<\/li>\n<li>  <code>key_name<\/code>\u0628\u0627 <code>key_alias<\/code>: \u0634\u0646\u0627\u0633\u0647 \u0647\u0627\u06cc \u0642\u0627\u0628\u0644 \u062e\u0648\u0627\u0646\u062f\u0646 \u0627\u0646\u0633\u0627\u0646\u06cc \u0628\u0631\u0627\u06cc \u06a9\u0644\u06cc\u062f.<\/li>\n<li>  <code>user_id<\/code>: \u0634\u0646\u0627\u0633\u0647 \u0628\u0631\u0627\u06cc \u06a9\u0627\u0631\u0628\u0631 \u062f\u0627\u062e\u0644\u06cc \u0645\u0631\u062a\u0628\u0637 \u0628\u0627 \u06a9\u0644\u06cc\u062f (\u062f\u0631 \u0635\u0648\u0631\u062a \u0648\u062c\u0648\u062f).<\/li>\n<li>  <code>team_id<\/code>: \u0634\u0646\u0627\u0633\u0647 \u0628\u0631\u0627\u06cc \u062a\u06cc\u0645 \u0645\u0631\u062a\u0628\u0637 \u0628\u0627 \u06a9\u0644\u06cc\u062f (\u062f\u0631 \u0635\u0648\u0631\u062a \u0648\u062c\u0648\u062f).<\/li>\n<li>  <code>org_id<\/code>: \u0634\u0646\u0627\u0633\u0647 \u0633\u0627\u0632\u0645\u0627\u0646 (\u062f\u0631 \u0635\u0648\u0631\u062a \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0648\u06cc\u0698\u06af\u06cc \u0647\u0627\u06cc org).<\/li>\n<li>  <code>spend<\/code>\u0628\u0627 <code>max_budget<\/code>\u0628\u0627 <code>soft_budget<\/code>\u0628\u0627 <code>budget_duration<\/code>\u0628\u0627 <code>expires<\/code>: \u0628\u0648\u062f\u062c\u0647 \u0648 \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0686\u0631\u062e\u0647 \u0639\u0645\u0631 \u0628\u0631\u0627\u06cc \u062e\u0648\u062f \u06a9\u0644\u06cc\u062f.<\/li>\n<li>  <code>tpm_limit<\/code>\u0628\u0627 <code>rpm_limit<\/code>\u0628\u0627 <code>max_parallel_requests<\/code>: \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0646\u0631\u062e \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0634\u062f\u0647 <em>\u0628\u0631\u0627\u06cc \u0627\u06cc\u0646 \u06a9\u0644\u06cc\u062f \u062e\u0627\u0635<\/em>\u0628\u0634\u0631<\/li>\n<li>  <code>models<\/code>: \u0644\u06cc\u0633\u062a\u06cc \u0627\u0632 \u0646\u0627\u0645 \u0647\u0627\/\u0627\u0644\u06af\u0648\u0647\u0627\u06cc \u0645\u062f\u0644 \u0627\u06cc\u0646 \u06a9\u0644\u06cc\u062f \u0645\u062c\u0627\u0632 \u0628\u0647 \u062f\u0633\u062a\u0631\u0633\u06cc \u0627\u0633\u062a.<\/li>\n<li>  <code>metadata<\/code>: \u0627\u0644\u0641 <strong>\u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0628\u062d\u0631\u0627\u0646\u06cc<\/strong> \u062c\u0627\u06cc\u06cc \u06a9\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0647\u0646\u06af\u0627\u0645 \u0627\u06cc\u062c\u0627\u062f\/\u0628\u0647 \u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc \u06a9\u0644\u06cc\u062f (\u0627\u0632 \u0637\u0631\u06cc\u0642 API \u06cc\u0627 UI) \u062c\u0641\u062a \u0647\u0627\u06cc \u0627\u0631\u0632\u0634 \u06a9\u0644\u06cc\u062f \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0630\u062e\u06cc\u0631\u0647 \u06a9\u0646\u06cc\u062f. <strong>\u0627\u0632 \u0627\u06cc\u0646 \u0628\u0647 \u0637\u0648\u0631 \u06af\u0633\u062a\u0631\u062f\u0647 \u0628\u0631\u0627\u06cc \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f<\/strong>\u060c \u0645\u0627\u0646\u0646\u062f \u0630\u062e\u06cc\u0631\u0647 \u0646\u0642\u0634 \u0647\u0627\u06cc \u06a9\u0627\u0631\u0628\u0631 \u060c \u067e\u0631\u0686\u0645 \u0647\u0627\u06cc \u0645\u062c\u0648\u0632 \u060c \u0642\u0648\u0627\u0646\u06cc\u0646 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u0633\u0641\u0627\u0631\u0634\u06cc \u060c \u063a\u0644\u0628\u0647 \u0628\u0631 \u0628\u0648\u062f\u062c\u0647 \u062e\u0627\u0635 \u0648 \u063a\u06cc\u0631\u0647.<\/li>\n<li>  <code>team_spend<\/code>\u0628\u0627 <code>team_max_budget<\/code>\u0628\u0627 <code>team_tpm_limit<\/code>\u0628\u0627 <code>team_rpm_limit<\/code>\u0628\u0627 <code>team_models<\/code>\u0628\u0627 <code>team_metadata<\/code>: \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0645\u0634\u0627\u0628\u0647 \u060c \u0627\u0645\u0627 \u0627\u0632 \u062a\u06cc\u0645 \u0645\u0631\u062a\u0628\u0637 \u0628\u0627 \u06a9\u0644\u06cc\u062f \u0628\u0647 \u0627\u0631\u062b \u0631\u0633\u06cc\u062f\u0647 \u0627\u0633\u062a. \u0645\u0646\u0637\u0642 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u062a\u0646\u0638\u06cc\u0645\u0627\u062a \u0633\u0637\u062d \u06a9\u0644\u06cc\u062f \u0631\u0627 \u062f\u0631 \u0633\u0637\u062d \u062a\u06cc\u0645 \u062f\u0631 \u0627\u0648\u0644\u0648\u06cc\u062a \u0642\u0631\u0627\u0631 \u062f\u0647\u062f \u06cc\u0627 \u0622\u0646\u0647\u0627 \u0631\u0627 \u0628\u0627 \u0647\u0645 \u062a\u0631\u06a9\u06cc\u0628 \u06a9\u0646\u062f.<\/li>\n<li>  <code>user_role<\/code>: \u0646\u0642\u0634 \u062a\u0639\u06cc\u06cc\u0646 \u0634\u062f\u0647 \u06a9\u0627\u0631\u0628\u0631 (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c <code>LitellmUserRoles.PROXY_ADMIN<\/code>\u0628\u0627 <code>LitellmUserRoles.INTERNAL_USER<\/code>).<\/li>\n<li>  <code>end_user_id<\/code>\u0628\u0627 <code>end_user_tpm_limit<\/code>\u0628\u0627 <code>end_user_rpm_limit<\/code>\u0628\u0627 <code>end_user_max_budget<\/code>: \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0645\u0631\u0628\u0648\u0637 \u0628\u0647 \u0634\u0646\u0627\u0633\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0646\u0647\u0627\u06cc\u06cc \u06a9\u0647 \u062f\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0645\u0646\u062a\u0642\u0644 \u0634\u062f\u0647 \u0627\u0633\u062a (<code>data['user']<\/code>) \u060c \u062f\u0631 \u0635\u0648\u0631\u062a \u0648\u062c\u0648\u062f \u0648 \u0631\u062f\u06cc\u0627\u0628\u06cc.<\/li>\n<li>  <code>parent_otel_span<\/code>: \u0628\u0631\u0627\u06cc \u0627\u062f\u063a\u0627\u0645 OpenTelemetry.<\/li>\n<li>  <em>&#8230; \u0648 \u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u0632\u0645\u06cc\u0646\u0647 \u0647\u0627\u06cc \u0628\u06cc\u0634\u062a\u0631\u06cc \u0628\u0633\u062a\u0647 \u0628\u0647 \u0646\u0633\u062e\u0647 Litellm \u0648 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc.<\/em><br \/>\n<strong>\u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0641\u0632\u0648\u0646\u0647:<\/strong> \u0628\u0631\u0627\u06cc \u0627\u062c\u0631\u0627\u06cc \u0647\u0631\u06af\u0648\u0646\u0647 \u0645\u0646\u0637\u0642\u06cc \u06a9\u0647 \u0628\u0647 \u0647\u0648\u06cc\u062a \u060c \u0645\u062c\u0648\u0632\u0647\u0627 \u060c \u0628\u0648\u062f\u062c\u0647 \u06cc\u0627 \u062a\u0646\u0638\u06cc\u0645\u0627\u062a \u0627\u0632 \u067e\u06cc\u0634 \u062a\u0646\u0638\u06cc\u0645 \u0634\u062f\u0647 \u0628\u0633\u062a\u06af\u06cc \u062f\u0627\u0631\u062f \u060c \u0636\u0631\u0648\u0631\u06cc \u0627\u0633\u062a.<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong><code>cache: DualCache<\/code><\/strong>: \u0646\u0645\u0648\u0646\u0647 \u0627\u06cc \u0627\u0632 <code>DualCache<\/code> \u06a9\u0644\u0627\u0633 (\u062a\u0639\u0631\u06cc\u0641 \u0634\u062f\u0647 \u062f\u0631 <code>caching\/dual_cache.py<\/code>). \u0627\u06cc\u0646 \u0634\u06cc\u0621 \u06cc\u06a9 \u0631\u0627\u0628\u0637 \u06a9\u0627\u0631\u0628\u0631\u06cc \u0628\u0631\u0627\u06cc \u0633\u06cc\u0633\u062a\u0645 \u0630\u062e\u06cc\u0631\u0647 \u0633\u0627\u0632\u06cc Litellm \u060c \u06a9\u0647 \u0628\u0647 \u0637\u0648\u0631 \u0645\u0639\u0645\u0648\u0644 \u06cc\u06a9 \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u062f\u0631 \u062d\u0627\u0641\u0638\u0647 \u0631\u0627 \u062a\u0631\u06a9\u06cc\u0628 \u0645\u06cc \u06a9\u0646\u062f (\u0645\u0627\u0646\u0646\u062f <code>InMemoryCache<\/code>) \u0628\u0631\u0627\u06cc \u0633\u0631\u0639\u062a \u0648 \u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u06cc\u06a9 \u062d\u0627\u0641\u0638\u0647 \u067e\u0646\u0647\u0627\u0646 \u062a\u0648\u0632\u06cc\u0639 \u0634\u062f\u0647 (\u0645\u0627\u0646\u0646\u062f <code>RedisCache<\/code>) \u0628\u0631\u0627\u06cc \u067e\u0627\u06cc\u062f\u0627\u0631\u06cc \u0648 \u0628\u0647 \u0627\u0634\u062a\u0631\u0627\u06a9 \u06af\u0630\u0627\u0631\u06cc \u0648\u0636\u0639\u06cc\u062a \u062f\u0631 \u0686\u0646\u062f\u06cc\u0646 \u0646\u0645\u0648\u0646\u0647 \u067e\u0631\u0648\u06a9\u0633\u06cc.<\/p>\n<ul>\n<li>  <strong>\u0631\u0648\u0634\u0647\u0627\u06cc \u06a9\u0644\u06cc\u062f\u06cc:<\/strong> <code>async_get_cache(key)<\/code>\u0628\u0627 <code>async_set_cache(key, value, ttl=...)<\/code>\u0628\u0627 <code>async_batch_get_cache(keys)<\/code>\u0628\u0627 <code>async_batch_set_cache(cache_list, ttl=...)<\/code>\u0628\u0627 <code>async_increment_cache(key, value, ttl=...)<\/code>\u0628\u0627 <code>async_delete_cache(key)<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0641\u0632\u0648\u0646\u0647:<\/strong> \u06a9\u0627\u0645\u0644\u0627\u064b \u0636\u0631\u0648\u0631\u06cc \u0628\u0631\u0627\u06cc <strong>\u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u062d\u0627\u0644\u062a<\/strong>\u0628\u0634\u0631 \u062f\u0631 <code>parallel_request_limiter.py<\/code> \u0645\u062b\u0627\u0644 \u0628\u0631\u0627\u06cc \u0630\u062e\u06cc\u0631\u0647 \u0648 \u0628\u0647 \u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc \u062a\u0639\u062f\u0627\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631 \u0647\u0631 \u06a9\u0644\u06cc\u062f\/\u06a9\u0627\u0631\u0628\u0631\/\u062a\u06cc\u0645 \u062f\u0631 \u0647\u0631 \u062f\u0642\u06cc\u0642\u0647 \u0628\u0647 \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u0645\u062a\u06a9\u06cc \u0627\u0633\u062a. \u0627\u0632 \u0622\u0646 \u0628\u0631\u0627\u06cc:\n<ul>\n<li>    \u067e\u06cc\u0634\u062e\u0648\u0627\u0646 \u0647\u0627\u06cc \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \/ \u067e\u0631\u062a\u0627\u0628.<\/li>\n<li>    \u0631\u062f\u06cc\u0627\u0628\u06cc \u0641\u0639\u0627\u0644\u06cc\u062a \u06a9\u0627\u0631\u0628\u0631 \u0627\u062e\u06cc\u0631.<\/li>\n<li>    \u0630\u062e\u06cc\u0631\u0647 \u067e\u0631\u0686\u0645 \u0647\u0627\u06cc \u0645\u0648\u0642\u062a \u06cc\u0627 \u062d\u0627\u0644\u062a \u0647\u0627\u06cc \u0645\u0631\u0628\u0648\u0637 \u0628\u0647 \u06a9\u0627\u0631\u0628\u0631 \u06cc\u0627 \u062c\u0631\u06cc\u0627\u0646 \u062f\u0631\u062e\u0648\u0627\u0633\u062a.<\/li>\n<li>    \u0630\u062e\u06cc\u0631\u0647 \u0633\u0627\u0632\u06cc \u0627\u0632 \u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u06a9\u0647 \u062a\u0648\u0633\u0637 \u0627\u0641\u0632\u0648\u0646\u0647 \u067e\u0631\u0633 \u0648 \u062c\u0648 \u0634\u062f\u0647 \u0627\u0633\u062a (\u0628\u0627 TTL \u0647\u0627\u06cc \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong><code>data: dict<\/code><\/strong>: \u0627\u06cc\u0646 \u0627\u0633\u062a <strong>\u0642\u0644\u0628 \u0628\u0627\u0631\u06af\u06cc\u0631\u06cc \u062f\u0631\u062e\u0648\u0627\u0633\u062a<\/strong> \u0622\u0646 Litellm \u062f\u0631 \u062d\u0627\u0644 \u0622\u0645\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0628\u0631\u0627\u06cc \u0627\u0631\u0633\u0627\u0644 \u0628\u0647 \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u0647 LLM \u0627\u0633\u0627\u0633\u06cc \u0627\u0633\u062a. <strong>\u0627\u0632 \u0646\u0638\u0631 \u0645\u0647\u0645 \u060c \u0642\u0627\u0628\u0644 \u062a\u063a\u06cc\u06cc\u0631 \u0627\u0633\u062a.<\/strong> \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a\u06cc \u06a9\u0647 \u062f\u0631 \u0627\u06cc\u0646 \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0627\u06cc\u062c\u0627\u062f \u0645\u06cc \u06a9\u0646\u06cc\u062f <em>\u062f\u0631\u0648\u0646 \u0642\u0644\u0627\u0628<\/em> \u062f\u0631 \u062a\u0645\u0627\u0633 \u0648\u0627\u0642\u0639\u06cc \u0633\u0627\u062e\u062a\u0647 \u0634\u062f\u0647 \u0645\u0646\u0639\u06a9\u0633 \u0645\u06cc \u0634\u0648\u062f \u060c <em>\u0627\u06af\u0631<\/em> \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 \u0627\u062c\u0627\u0632\u0647 \u0645\u06cc \u062f\u0647\u062f \u062a\u0627 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u062f\u0627\u0645\u0647 \u06cc\u0627\u0628\u062f.<\/p>\n<ul>\n<li>  <strong>\u0645\u062d\u062a\u0648\u0627:<\/strong> \u0634\u0627\u0645\u0644 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u062a\u0645\u0627\u0633 LLM \u0645\u0627\u0646\u0646\u062f <code>model<\/code>\u0628\u0627 <code>messages<\/code>\u0628\u0627 <code>input<\/code> (\u0628\u0631\u0627\u06cc \u062a\u0639\u0628\u06cc\u0647) \u060c <code>temperature<\/code>\u0628\u0627 <code>max_tokens<\/code>\u0628\u0627 <code>stream<\/code>\u0628\u0627 <code>tools<\/code>\u0628\u0627 <code>function_call<\/code>\u0628\u0627 <code>user<\/code>\u060c \u0648 \u063a\u06cc\u0631\u0647<\/li>\n<li>  <strong>\u063a\u0646\u06cc \u0633\u0627\u0632\u06cc:<\/strong> \u0647\u0645\u0627\u0646\u0637\u0648\u0631 \u06a9\u0647 \u062a\u0648\u0633\u0637 <code>litellm_pre_call_utils.py<\/code>\u060c \u0627\u06cc\u0646 \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0627\u0632 \u0642\u0628\u0644 \u0628\u0647 \u0642\u0644\u0627\u0628 \u0645\u06cc \u0631\u0633\u062f <strong>\u063a\u0646\u06cc \u0634\u062f\u0647<\/strong> \u0628\u0627 <code>metadata<\/code> (\u06cc\u0627 <code>litellm_metadata<\/code>) \u0632\u06cc\u0631 \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u062d\u0627\u0648\u06cc \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0645\u062a\u0646\u06cc \u062d\u0627\u0635\u0644 \u0627\u0632 <code>user_api_key_dict<\/code> \u0648 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u062f\u0631\u062e\u0648\u0627\u0633\u062a\/\u067e\u0631\u0633 \u0648 \u062c\u0648. \u0627\u06cc\u0646 \u0628\u062f\u0627\u0646 \u0645\u0639\u0646\u0627\u0633\u062a \u06a9\u0647 \u0634\u0645\u0627 \u0627\u063a\u0644\u0628 \u0646\u06cc\u0627\u0632\u06cc \u0628\u0647 \u0645\u062c\u062f\u062f\u0627\u064b \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0627\u0633\u0627\u0633\u06cc \u0627\u0632 \u0622\u0646 \u0646\u062f\u0627\u0631\u06cc\u062f <code>user_api_key_dict<\/code> \u0627\u06af\u0631 \u0627\u0632 \u0642\u0628\u0644 \u0628\u0647 \u0631\u0627\u062d\u062a\u06cc \u0642\u0631\u0627\u0631 \u062f\u0627\u062f\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a <code>data['metadata']<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0641\u0632\u0648\u0646\u0647:<\/strong>\n<ul>\n<li>  <strong>\u0628\u0627\u0632\u0631\u0633\u06cc:<\/strong> \u0645\u0642\u0627\u062f\u06cc\u0631 \u0645\u0627\u0646\u0646\u062f \u0631\u0627 \u0628\u062e\u0648\u0627\u0646\u06cc\u062f <code>data['model']<\/code>\u0628\u0627 <code>data['messages']<\/code>\u0628\u0627 <code>data['user']<\/code> \u0628\u0631\u0627\u06cc \u062a\u0635\u0645\u06cc\u0645 \u06af\u06cc\u0631\u06cc<\/li>\n<li>  <strong>\u0627\u0635\u0644\u0627\u062d:<\/strong> \u0628\u0647 \u0637\u0648\u0631 \u0645\u0633\u062a\u0642\u06cc\u0645 \u0645\u0642\u0627\u062f\u06cc\u0631 \u0631\u0627 \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0647\u06cc\u062f: <code>data['model'] = 'new-model'<\/code>\u0628\u0627 <code>data['max_tokens'] = 500<\/code>\u0628\u0627 <code>data['messages'].append(...)<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u0639\u0644\u0627\u0648\u0647 \u0628\u0631 \u0627\u06cc\u0646:<\/strong> \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u062c\u062f\u06cc\u062f \u0631\u0627 \u062f\u0631 \u0635\u0648\u0631\u062a \u067e\u0634\u062a\u06cc\u0628\u0627\u0646\u06cc \u062a\u0648\u0633\u0637 \u0645\u062f\u0644 \u0647\u062f\u0641\/LITELLM \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u062f: <code>data['custom_param'] = 'value'<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u062d\u0630\u0641:<\/strong> \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627 \u0631\u0627 \u062d\u0630\u0641 \u06a9\u0646\u06cc\u062f: <code>del data['frequency_penalty']<\/code>\u0628\u0634\u0631<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong><code>call_type: Literal[...]<\/code><\/strong>: \u0631\u0634\u062a\u0647 \u0627\u06cc \u06a9\u0647 \u0646\u0634\u0627\u0646 \u062f\u0647\u0646\u062f\u0647 \u0646\u0648\u0639 \u0639\u0645\u0644\u06cc\u0627\u062a LLM \u0627\u0633\u062a. \u0627\u06cc\u0646 \u0628\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0627\u062c\u0627\u0632\u0647 \u0645\u06cc \u062f\u0647\u062f \u062a\u0627 \u0645\u0646\u0637\u0642 \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u0627\u0646\u062a\u062e\u0627\u0628\u06cc \u0627\u0639\u0645\u0627\u0644 \u06a9\u0646\u062f.<\/p>\n<ul>\n<li>  <strong>\u0645\u062b\u0627\u0644\u0647\u0627:<\/strong> <code>\"completion\"<\/code>\u0628\u0627 <code>\"embeddings\"<\/code>\u0628\u0627 <code>\"image_generation\"<\/code>\u0628\u0627 <code>\"moderation\"<\/code>\u0628\u0627 <code>\"audio_transcription\"<\/code>\u0628\u0627 <code>\"rerank\"<\/code>\u0628\u0627 <code>\"pass_through_endpoint\"<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u0641\u0632\u0648\u0646\u0647:<\/strong> \u0627\u0632 \u0645\u0646\u0637\u0642 \u0634\u0631\u0637\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f (<code>if call_type == \"completion\": ...<\/code>) \u0628\u0631\u0627\u06cc \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u0627\u0632 \u0627\u06cc\u0646 \u06a9\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0641\u0642\u0637 \u0628\u0631\u0627\u06cc \u0639\u0645\u0644\u06cc\u0627\u062a \u0645\u0648\u0631\u062f \u0646\u0638\u0631 \u0628\u0647 \u062f\u0631\u0633\u062a\u06cc \u0627\u062c\u0631\u0627 \u0645\u06cc \u0634\u0648\u062f \u06cc\u0627 \u0631\u0641\u062a\u0627\u0631 \u0645\u06cc \u06a9\u0646\u062f. \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u06cc\u06a9 \u0627\u0641\u0632\u0648\u0646\u0647 \u0627\u0635\u0644\u0627\u062d \u0633\u0631\u06cc\u0639 \u0628\u0627\u06cc\u062f \u0641\u0642\u0637 \u0628\u0631\u0627\u06cc \u0622\u0646 \u0627\u062c\u0631\u0627 \u0634\u0648\u062f <code>\"completion\"<\/code>\u0628\u0634\u0631 \u0627\u0641\u0632\u0648\u0646\u0647 \u0686\u06a9 \u0647\u0632\u06cc\u0646\u0647 \u062a\u0639\u0628\u06cc\u0647 \u0634\u062f\u0647 \u0641\u0642\u0637 \u0628\u0627\u06cc\u062f \u0628\u0631\u0627\u06cc \u0622\u0646 \u0627\u062c\u0631\u0627 \u0634\u0648\u062f <code>\"embeddings\"<\/code>\u0628\u0634\u0631<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>\u0645\u0642\u062f\u0627\u0631 \u0628\u0627\u0632\u06af\u0634\u062a (<code>-&gt; Optional[Union[Exception, str, dict]]<\/code>)<\/strong>: \u0627\u06cc\u0646 \u0633\u0631\u0646\u0648\u0634\u062a \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0631\u0627 \u067e\u0633 \u0627\u0632 \u0627\u062a\u0645\u0627\u0645 \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 \u062a\u0639\u06cc\u06cc\u0646 \u0645\u06cc \u06a9\u0646\u062f.<\/p>\n<ul>\n<li>  <strong>\u0628\u0627\u0632\u06af\u0634\u062a <code>None<\/code> (\u0628\u0647 \u0637\u0648\u0631 \u0636\u0645\u0646\u06cc \u06cc\u0627 \u0635\u0631\u06cc\u062d):<\/strong> <strong>\u0627\u062f\u0627\u0645\u0647 \u062f\u0647\u06cc\u062f<\/strong> \u0633\u06cc\u06af\u0646\u0627\u0644 \u0647\u0627\u06cc\u06cc \u06a9\u0647 \u0686\u06a9 \u0647\u0627\u06cc \u0647\u0648\u06a9 \u0628\u0647 \u062a\u0635\u0648\u06cc\u0628 \u0631\u0633\u06cc\u062f\u0647 \u0648 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0627\u06cc\u062f \u0628\u0647 \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u0647 LLM \u0627\u062f\u0627\u0645\u0647 \u06cc\u0627\u0628\u062f. \u062f\u0631 <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a (\u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u062f\u0631 \u0642\u0644\u0627\u0628 \u0627\u0635\u0644\u0627\u062d \u0634\u062f\u0647) \u0628\u0631\u0627\u06cc \u062a\u0645\u0627\u0633 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u0634\u0648\u062f. <em>\u0627\u06cc\u0646 \u0631\u0648\u0634 \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u0628\u0631\u0627\u06cc \u0627\u062c\u0627\u0632\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u067e\u0633 \u0627\u0632 \u0628\u0627\u0632\u0631\u0633\u06cc\/\u0627\u0635\u0644\u0627\u062d \u0627\u0633\u062a.<\/em>\n<\/li>\n<li>  <strong>\u0628\u0627\u0632\u06af\u0634\u062a <code>data<\/code> (<code>dict<\/code>):<\/strong> <strong>\u0628\u0627 \u0627\u0635\u0644\u0627\u062d\u0627\u062a \u0627\u062f\u0627\u0645\u0647 \u062f\u0647\u06cc\u062f.<\/strong> \u0635\u0631\u06cc\u062d\u0627\u064b (\u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u0627\u0635\u0644\u0627\u062d \u0634\u062f\u0647) \u0631\u0627 \u0628\u0631\u0645\u06cc \u06af\u0631\u062f\u0627\u0646\u062f <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0627\u0632 \u0646\u0638\u0631 \u0639\u0645\u0644\u06a9\u0631\u062f\u06cc \u0634\u0628\u06cc\u0647 \u0628\u0647 \u0628\u0627\u0632\u06af\u0634\u062a <code>None<\/code> \u067e\u0633 \u0627\u0632 \u0627\u0635\u0644\u0627\u062d <code>data<\/code> \u062f\u0631 \u062c\u0627\u06cc \u062e\u0648\u062f \u060c \u0627\u0645\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u062f \u0648\u0627\u0636\u062d \u062a\u0631 \u0628\u0627\u0634\u062f. \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0628\u0631\u06af\u0634\u062a\u06cc <em>\u062a\u0639\u0648\u06cc\u0636<\/em> \u0627\u0635\u0644\u06cc <code>data<\/code> \u0628\u0631\u0627\u06cc \u062a\u0645\u0627\u0633 LLM<\/li>\n<li>  <strong>\u0628\u0627\u0632\u06af\u0634\u062a <code>str<\/code>:<\/strong> <strong>\u0628\u0627 \u067e\u0627\u0633\u062e \u0633\u0641\u0627\u0631\u0634\u06cc (\u0641\u0642\u0637 \u0686\u062a\/\u062a\u06a9\u0645\u06cc\u0644) \u0631\u062f \u06a9\u0646\u06cc\u062f.<\/strong> \u0628\u0631\u0627\u06cc <code>call_type<\/code> &#8220;\u062a\u06a9\u0645\u06cc\u0644&#8221; \u06cc\u0627 &#8220;text_completion&#8221; \u060c \u0628\u0627\u0632\u06af\u0634\u062a \u0628\u0647 \u0631\u062f \u0633\u06cc\u06af\u0646\u0627\u0644 \u0647\u0627\u06cc \u0631\u0634\u062a\u0647. Litellm \u0627\u06cc\u0646 \u0631\u0627 \u0631\u0647\u06af\u06cc\u0631\u06cc \u0645\u06cc \u06a9\u0646\u062f \u060c <em>LLM \u0631\u0627 \u0635\u062f\u0627 \u0646\u0645\u06cc \u06a9\u0646\u062f<\/em>\u060c \u0648 \u0631\u0634\u062a\u0647 \u0631\u0627 \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 \u067e\u0627\u0633\u062e \u0645\u0648\u0641\u0642\u06cc\u062a \u0622\u0645\u06cc\u0632 \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u0627\u0632 \u062f\u0633\u062a\u06cc\u0627\u0631 (\u0647\u0645\u0627\u0646\u0637\u0648\u0631 \u06a9\u0647 \u062f\u0631 \u0645\u062b\u0627\u0644 \u0631\u062f &#8220;\u0633\u0644\u0627\u0645 \u062c\u0647\u0627\u0646&#8221; \u0645\u0634\u0627\u0647\u062f\u0647 \u0645\u06cc \u0634\u0648\u062f) \u0642\u0627\u0644\u0628 \u0628\u0646\u062f\u06cc \u0645\u06cc \u06a9\u0646\u062f. \u0628\u0631\u0627\u06cc \u062f\u06cc\u06af\u0631\u06cc <code>call_type<\/code>S \u060c \u0627\u06cc\u0646 \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u0645\u0646\u062c\u0631 \u0628\u0647 \u062e\u0637\u0627\u06cc 400 \u06cc\u0627 \u0631\u0641\u062a\u0627\u0631 \u063a\u06cc\u0631 \u0645\u0646\u062a\u0638\u0631\u0647 \u0634\u0648\u062f &#8211; \u062f\u0631 \u062f\u0631\u062c\u0647 \u0627\u0648\u0644 \u0627\u0632 \u0627\u06cc\u0646 \u0628\u0631\u0627\u06cc \u0646\u0642\u0627\u0637 \u067e\u0627\u06cc\u0627\u0646\u06cc \u0686\u062a \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u0628\u0627\u0644\u0627 \u0628\u0631\u062f\u0646 <code>Exception<\/code>:<\/strong> <strong>\u0628\u0627 \u062e\u0637\u0627 \u0631\u062f \u06a9\u0646\u06cc\u062f.<\/strong> \u0627\u06cc\u0646 \u0631\u0648\u0634 \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u0628\u0631\u0627\u06cc \u0631\u062f \u0632\u0648\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0647 \u062f\u0644\u06cc\u0644 \u0646\u0642\u0636 \u062e\u0637 \u0645\u0634\u06cc \u060c \u0648\u0631\u0648\u062f\u06cc \u0646\u0627\u0645\u0639\u062a\u0628\u0631 \u060c \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0628\u0648\u062f\u062c\u0647 \u06cc\u0627 \u0686\u06a9 \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u0634\u06a9\u0633\u062a \u062e\u0648\u0631\u062f\u0647 \u0627\u0633\u062a.\n<ul>\n<li>    \u0627\u0641\u0632\u0627\u06cc\u0634 <code>fastapi.HTTPException(status_code=..., detail=...)<\/code> \u0628\u0633\u06cc\u0627\u0631 \u062a\u0648\u0635\u06cc\u0647 \u0645\u06cc \u0634\u0648\u062f \u0627\u06cc\u0646 \u0627\u0645\u06a9\u0627\u0646 \u0631\u0627 \u0628\u0647 \u0634\u0645\u0627 \u0645\u06cc \u062f\u0647\u062f \u062a\u0627 \u06a9\u062f \u062f\u0642\u06cc\u0642 \u0648\u0636\u0639\u06cc\u062a HTTP \u0631\u0627 \u06a9\u0646\u062a\u0631\u0644 \u06a9\u0646\u06cc\u062f (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c 400 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u062f \u060c 401 \u063a\u06cc\u0631\u0645\u062c\u0627\u0632 \u060c 403 \u0645\u0645\u0646\u0648\u0639 \u060c 429 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u06cc\u0634 \u0627\u0632 \u062d\u062f \u060c 500 \u062e\u0637\u0627\u06cc \u0633\u0631\u0648\u0631 \u062f\u0627\u062e\u0644\u06cc) \u0648 \u067e\u06cc\u0627\u0645 \u062e\u0637\u0627 \u0628\u0647 \u0645\u0634\u062a\u0631\u06cc \u0627\u0635\u0644\u06cc \u0628\u0627\u0632\u06af\u0634\u062a.<\/li>\n<li>    \u062f\u0631 <code>parallel_request_limiter.py<\/code> \u06a9\u0627\u0631\u0628\u0631\u062f\u0647\u0627\u06cc <code>HTTPException(status_code=429, ...)<\/code> \u0628\u0631\u0627\u06cc \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0646\u0631\u062e \u0628\u06cc\u0634 \u0627\u0632 \u062e\u0637\u0627\u0647\u0627\u06cc.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_3_%D8%A7%D8%AC%D8%B1%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D8%AE%D9%88%D8%AF-%DA%AF%D8%A7%D9%85_%D8%A8%D9%87_%DA%AF%D8%A7%D9%85\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 3: \u0627\u062c\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f-\u06af\u0627\u0645 \u0628\u0647 \u06af\u0627\u0645<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0631\u0648\u0646\u062f \u0627\u06cc\u062c\u0627\u062f \u06cc\u06a9 \u0627\u0641\u0632\u0648\u0646\u0647 \u0633\u0627\u062f\u0647 \u0631\u0627 \u0637\u06cc \u06a9\u0646\u06cc\u0645. \u0645\u0627 \u06cc\u06a9 \u0627\u0641\u0632\u0648\u0646\u0647 \u0627\u06cc\u062c\u0627\u062f \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f \u06a9\u0647:<\/p>\n<ol>\n<li>  \u0628\u0631\u0631\u0633\u06cc \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0622\u06cc\u0627 \u06a9\u0627\u0631\u0628\u0631 \u0646\u0642\u0634 \u062e\u0627\u0635\u06cc \u0631\u0627 \u062f\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u0627\u0635\u0644\u06cc \u062e\u0648\u062f \u062a\u0639\u0631\u06cc\u0641 \u06a9\u0631\u062f\u0647 \u0627\u0633\u062a (<code>plugin_access: true<\/code>).<\/li>\n<li>  \u0627\u06af\u0631 \u0622\u0646\u0647\u0627 \u062f\u0633\u062a\u0631\u0633\u06cc \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u0646\u062f \u0648 \u06cc\u06a9 \u062a\u0645\u0627\u0633 &#8220;\u062a\u06a9\u0645\u06cc\u0644&#8221; \u0631\u0627 \u0627\u0646\u062c\u0627\u0645 \u0645\u06cc \u062f\u0647\u0646\u062f \u060c \u062d\u062f\u0627\u06a9\u062b\u0631 \u0631\u0627 \u062a\u0642\u0648\u06cc\u062a \u0645\u06cc \u06a9\u0646\u062f <code>max_tokens<\/code> \u0645\u062d\u062f\u0648\u062f \u06a9\u0631\u062f\u0646<\/li>\n<li>  \u0627\u06af\u0631 \u0622\u0646\u0647\u0627 \u062f\u0633\u062a\u0631\u0633\u06cc \u0646\u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u0646\u062f \u060c \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0631\u0627 \u0628\u0627 \u062e\u0637\u0627\u06cc \u0645\u0645\u0646\u0648\u0639\u0647 403 \u0631\u062f \u0645\u06cc \u06a9\u0646\u062f.<\/li>\n<\/ol>\n<p><strong>\u0645\u0631\u062d\u0644\u0647 1: \u067e\u0631\u0648\u0646\u062f\u0647 \u06a9\u0646\u062a\u0631\u0644 \u06a9\u0646\u0646\u062f\u0647 \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f<\/strong><\/p>\n<p>\u06cc\u06a9 \u0641\u0627\u06cc\u0644 \u067e\u0627\u06cc\u062a\u0648\u0646 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c <code>my_plugins.py<\/code>) \u062c\u0627\u06cc\u06cc \u06a9\u0647 \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0634\u0645\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u062f \u0622\u0646 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u062f.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"c1\"># my_plugins.py\n<\/span><span class=\"kn\">import<\/span> <span class=\"n\">sys<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">litellm.integrations.custom_logger<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">CustomLogger<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">litellm.proxy._types<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">UserAPIKeyAuth<\/span> <span class=\"c1\"># Import the type hint\n<\/span><span class=\"kn\">from<\/span> <span class=\"n\">litellm.caching.dual_cache<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">DualCache<\/span> <span class=\"c1\"># Import the type hint\n<\/span><span class=\"kn\">from<\/span> <span class=\"n\">typing<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Optional<\/span><span class=\"p\">,<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">,<\/span> <span class=\"n\">Union<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">fastapi<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">HTTPException<\/span> <span class=\"c1\"># Import for raising specific errors\n<\/span><span class=\"kn\">import<\/span> <span class=\"n\">litellm<\/span> <span class=\"c1\"># Optional, but good practice\n<\/span>\n<span class=\"c1\"># Define your plugin class, inheriting from CustomLogger\n<\/span><span class=\"k\">class<\/span> <span class=\"nc\">AccessAndTokenLimitPlugin<\/span><span class=\"p\">(<\/span><span class=\"n\">CustomLogger<\/span><span class=\"p\">):<\/span>\n\n    <span class=\"k\">def<\/span> <span class=\"nf\">__init__<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"n\">max_tokens_limit<\/span><span class=\"p\">:<\/span> <span class=\"nb\">int<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1024<\/span><span class=\"p\">):<\/span>\n        <span class=\"sh\">\"\"\"<\/span><span class=\"s\">\n        Initialize the plugin, potentially with configuration.\n        <\/span><span class=\"sh\">\"\"\"<\/span>\n        <span class=\"nf\">super<\/span><span class=\"p\">().<\/span><span class=\"nf\">__init__<\/span><span class=\"p\">()<\/span> <span class=\"c1\"># Call the base class initializer\n<\/span>        <span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span> <span class=\"o\">=<\/span> <span class=\"n\">max_tokens_limit<\/span>\n        <span class=\"nf\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">AccessAndTokenLimitPlugin Initialized with max_tokens=<\/span><span class=\"si\">{<\/span><span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"c1\"># Implement the async_pre_call_hook method\n<\/span>    <span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span>\n        <span class=\"n\">self<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">:<\/span> <span class=\"n\">UserAPIKeyAuth<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">cache<\/span><span class=\"p\">:<\/span> <span class=\"n\">DualCache<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[<\/span>\n            <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">text_completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">embeddings<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span>\n            <span class=\"sh\">\"<\/span><span class=\"s\">image_generation<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">moderation<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">audio_transcription<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span>\n            <span class=\"sh\">\"<\/span><span class=\"s\">pass_through_endpoint<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">rerank<\/span><span class=\"sh\">\"<\/span> <span class=\"c1\"># Add other relevant types\n<\/span>        <span class=\"p\">],<\/span>\n    <span class=\"p\">)<\/span> <span class=\"o\">-&gt;<\/span> <span class=\"n\">Optional<\/span><span class=\"p\">[<\/span><span class=\"n\">Union<\/span><span class=\"p\">[<\/span><span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"nb\">str<\/span><span class=\"p\">]]:<\/span> <span class=\"c1\"># Return type annotation (raising Exception is also an outcome)\n<\/span>        <span class=\"sh\">\"\"\"<\/span><span class=\"s\">\n        This hook checks user access via metadata and enforces token limits.\n        <\/span><span class=\"sh\">\"\"\"<\/span>\n        <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">------ AccessAndTokenLimitPlugin Start ------<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Call Type: <\/span><span class=\"si\">{<\/span><span class=\"n\">call_type<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">User\/Key Info: <\/span><span class=\"si\">{<\/span><span class=\"n\">user_api_key_dict<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"c1\"># Be careful logging sensitive info in prod\n<\/span>\n        <span class=\"c1\"># --- 1. Access Control ---\n<\/span>        <span class=\"c1\"># Check for 'plugin_access: true' in the key's metadata\n<\/span>        <span class=\"n\">key_metadata<\/span> <span class=\"o\">=<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">metadata<\/span> <span class=\"ow\">or<\/span> <span class=\"p\">{}<\/span> <span class=\"c1\"># Safely get metadata\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">key_metadata<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">plugin_access<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"ow\">is<\/span> <span class=\"ow\">not<\/span> <span class=\"bp\">True<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Access Denied: Key metadata missing <\/span><span class=\"sh\">'<\/span><span class=\"s\">plugin_access: true<\/span><span class=\"sh\">'<\/span><span class=\"s\">. Metadata: <\/span><span class=\"si\">{<\/span><span class=\"n\">key_metadata<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n            <span class=\"c1\"># Reject with 403 Forbidden\n<\/span>            <span class=\"k\">raise<\/span> <span class=\"nc\">HTTPException<\/span><span class=\"p\">(<\/span>\n                <span class=\"n\">status_code<\/span><span class=\"o\">=<\/span><span class=\"mi\">403<\/span><span class=\"p\">,<\/span> <span class=\"c1\"># Forbidden\n<\/span>                <span class=\"n\">detail<\/span><span class=\"o\">=<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Access Denied: Your API key does not have permission for this operation via the AccessAndTokenLimitPlugin.<\/span><span class=\"sh\">\"<\/span>\n            <span class=\"p\">)<\/span>\n\n        <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Access Granted: <\/span><span class=\"sh\">'<\/span><span class=\"s\">plugin_access: true<\/span><span class=\"sh\">'<\/span><span class=\"s\"> found.<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"c1\"># --- 2. Enforce Max Tokens (Only for Completion Calls) ---\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">call_type<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">original_max_tokens<\/span> <span class=\"o\">=<\/span> <span class=\"n\">data<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">max_tokens<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n            <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Original max_tokens: <\/span><span class=\"si\">{<\/span><span class=\"n\">original_max_tokens<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n            <span class=\"k\">if<\/span> <span class=\"n\">original_max_tokens<\/span> <span class=\"ow\">is<\/span> <span class=\"bp\">None<\/span> <span class=\"ow\">or<\/span> <span class=\"n\">original_max_tokens<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span><span class=\"p\">:<\/span>\n                <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Enforcing max_tokens limit: Setting to <\/span><span class=\"si\">{<\/span><span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n                <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">\"<\/span><span class=\"s\">max_tokens<\/span><span class=\"sh\">\"<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span> <span class=\"c1\"># Modify the data dictionary directly\n<\/span>            <span class=\"k\">else<\/span><span class=\"p\">:<\/span>\n                 <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Existing max_tokens (<\/span><span class=\"si\">{<\/span><span class=\"n\">original_max_tokens<\/span><span class=\"si\">}<\/span><span class=\"s\">) is within limit (<\/span><span class=\"si\">{<\/span><span class=\"n\">self<\/span><span class=\"p\">.<\/span><span class=\"n\">max_tokens_limit<\/span><span class=\"si\">}<\/span><span class=\"s\">).<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"c1\"># --- 3. Allow Request ---\n<\/span>        <span class=\"c1\"># If all checks pass, allow the request to proceed.\n<\/span>        <span class=\"c1\"># Returning None implicitly allows the request with any modifications made to 'data'.\n<\/span>        <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">------ AccessAndTokenLimitPlugin End (Allowing) ------<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span> <span class=\"c1\"># Or return data\n<\/span>\n<span class=\"c1\"># Create an instance of your plugin handler\n# You can pass configuration here if needed\n<\/span><span class=\"n\">plugin_handler_instance<\/span> <span class=\"o\">=<\/span> <span class=\"nc\">AccessAndTokenLimitPlugin<\/span><span class=\"p\">(<\/span><span class=\"n\">max_tokens_limit<\/span><span class=\"o\">=<\/span><span class=\"mi\">512<\/span><span class=\"p\">)<\/span>\n\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><strong>\u0645\u0631\u062d\u0644\u0647 2: \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0631\u0627 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc \u06a9\u0646\u06cc\u062f (<code>config.yaml<\/code>)<\/strong><\/p>\n<p>\u067e\u0631\u0648\u06a9\u0633\u06cc \u062e\u0648\u062f \u0631\u0627 \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0647\u06cc\u062f <code>config.yaml<\/code> \u0628\u0631\u0627\u06cc \u0627\u06cc\u0646\u06a9\u0647 \u0628\u0647 Litellm \u0628\u06af\u0648\u06cc\u06cc\u062f \u0627\u0632 \u06a9\u0646\u062a\u0631\u0644 \u06a9\u0646\u0646\u062f\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u062c\u062f\u06cc\u062f \u062e\u0648\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u062f.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"c1\"># config.yaml<\/span>\n\n<span class=\"na\">model_list<\/span><span class=\"pi\">:<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"na\">model_name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">gpt-3.5-turbo<\/span>\n    <span class=\"na\">litellm_params<\/span><span class=\"pi\">:<\/span>\n      <span class=\"na\">model<\/span><span class=\"pi\">:<\/span> <span class=\"s\">openai\/gpt-3.5-turbo<\/span>\n  <span class=\"c1\"># Add other models as needed<\/span>\n\n<span class=\"na\">litellm_settings<\/span><span class=\"pi\">:<\/span>\n  <span class=\"c1\"># Register your plugin instance(s) here<\/span>\n  <span class=\"c1\"># Make sure the path (my_plugins.plugin_handler_instance) is correct<\/span>\n  <span class=\"c1\"># relative to where you run the `litellm` command.<\/span>\n  <span class=\"na\">callbacks<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span><span class=\"nv\">my_plugins.plugin_handler_instance<\/span><span class=\"pi\">]<\/span>\n  <span class=\"c1\"># You can add other callbacks too:<\/span>\n  <span class=\"c1\"># success_callback: [\"langfuse\", \"my_other_logger.instance\"]<\/span>\n  <span class=\"na\">set_verbose<\/span><span class=\"pi\">:<\/span> <span class=\"kc\">true<\/span> <span class=\"c1\"># Recommended for debugging plugins<\/span>\n\n<span class=\"na\">general_settings<\/span><span class=\"pi\">:<\/span>\n  <span class=\"c1\"># Add any other general settings<\/span>\n  <span class=\"na\">master_key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">sk-1234<\/span> <span class=\"c1\"># Example, use a real key management strategy<\/span>\n\n<span class=\"c1\"># Example key definitions (if not using DB\/UI)<\/span>\n<span class=\"c1\"># Ensure keys used for testing have the required metadata<\/span>\n<span class=\"na\">keys<\/span><span class=\"pi\">:<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"na\">key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">sk-plugin-allowed-key<\/span>\n    <span class=\"na\">metadata<\/span><span class=\"pi\">:<\/span>\n      <span class=\"na\">plugin_access<\/span><span class=\"pi\">:<\/span> <span class=\"kc\">true<\/span> <span class=\"c1\"># This key *should* pass the plugin check<\/span>\n      <span class=\"na\">user_id<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">user-allowed\"<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"na\">key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">sk-plugin-denied-key<\/span>\n    <span class=\"na\">metadata<\/span><span class=\"pi\">:<\/span>\n      <span class=\"c1\"># plugin_access is missing or false<\/span>\n      <span class=\"na\">user_id<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">user-denied\"<\/span>\n  <span class=\"pi\">-<\/span> <span class=\"na\">key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">sk-no-metadata-key<\/span>\n    <span class=\"na\">user_id<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">user-no-meta\"<\/span> <span class=\"c1\"># This key will also be denied by the plugin<\/span>\n\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><strong>\u0645\u0631\u062d\u0644\u0647 3: \u067e\u0631\u0648\u06a9\u0633\u06cc \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f \u0648 \u062a\u0633\u062a \u06a9\u0646\u06cc\u062f<\/strong><\/p>\n<ol>\n<li> <strong>\u0630\u062e\u06cc\u0631\u0647 \u067e\u0631\u0648\u0646\u062f\u0647 \u0647\u0627:<\/strong> \u062a\u0636\u0645\u06cc\u0646 \u06a9\u0631\u062f\u0646 <code>my_plugins.py<\/code> \u0648\u062a <code>config.yaml<\/code> \u0628\u0647 \u062f\u0631\u0633\u062a\u06cc \u0630\u062e\u06cc\u0631\u0647 \u0645\u06cc \u0634\u0648\u0646\u062f<\/li>\n<li>\n<p><strong>\u067e\u0631\u0648\u06a9\u0633\u06cc \u0631\u0627 \u0634\u0631\u0648\u0639 \u06a9\u0646\u06cc\u062f:<\/strong> \u067e\u0631\u0648\u06a9\u0633\u06cc \u0631\u0627 \u0627\u0632 \u062a\u0631\u0645\u06cc\u0646\u0627\u0644 \u062e\u0648\u062f \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f \u060c \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u062d\u0627\u0635\u0644 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u062f \u067e\u0631\u0648\u0646\u062f\u0647 \u0647\u0627\u06cc \u0634\u0645\u0627 \u0631\u0627 \u067e\u06cc\u062f\u0627 \u06a9\u0646\u062f.<\/p>\n<pre class=\"highlight shell\"><code>litellm <span class=\"nt\">--config<\/span> config.yaml <span class=\"nt\">--logs<\/span>\n<\/code><\/pre>\n<\/li>\n<li>\n<p><strong>\u0645\u0648\u0631\u062f \u0622\u0632\u0645\u0648\u0646 1: \u06a9\u0627\u0631\u0628\u0631 \u0645\u062c\u0627\u0632 (\u062d\u062f\u0627\u06a9\u062b\u0631 \u0646\u0634\u0627\u0646\u0647 \u0647\u0627\u06cc \u0627\u062c\u0628\u0627\u0631\u06cc)<\/strong><\/p>\n<ul>\n<li>    \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>sk-plugin-allowed-key<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u06cc\u06a9 <code>\/chat\/completions<\/code> \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062a\u0645\u0627\u0633 <em>\u0628\u06cc\u0634\u062a\u0631<\/em> \u0627\u0632 \u062d\u062f \u0627\u0641\u0632\u0648\u0646\u0647 (512 \u062f\u0631 \u0645\u062b\u0627\u0644 \u0645\u0627).\n<\/li>\n<\/ul>\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> POST http:\/\/localhost:4000\/chat\/completions <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Content-Type: application\/json\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Authorization: Bearer sk-plugin-allowed-key\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-d<\/span> <span class=\"s1\">'{\n    \"model\": \"gpt-3.5-turbo\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story.\"}],\n    \"max_tokens\": 1000\n  }'<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ol>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>*   **Expected Outcome:** The request should succeed. Check the proxy logs for `AccessAndTokenLimitPlugin` messages. You should see \"Access Granted\" and \"Enforcing max_tokens limit: Setting to 512\". The actual LLM call will use `max_tokens: 512`.\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<ol>\n<li>\n<p><strong>\u0645\u0648\u0631\u062f \u0622\u0632\u0645\u0627\u06cc\u0634 2: \u06a9\u0627\u0631\u0628\u0631 \u0645\u062c\u0627\u0632 (\u062d\u062f\u0627\u06a9\u062b\u0631 \u0646\u0634\u0627\u0646\u0647 \u0647\u0627 \u062f\u0631 \u062d\u062f \u0645\u062c\u0627\u0632)<\/strong><\/p>\n<ul>\n<li>    \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>sk-plugin-allowed-key<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u062f\u0631\u062e\u0648\u0627\u0633\u062a <code>max_tokens<\/code> <em>\u06a9\u0645\u062a\u0631<\/em> \u0627\u0632 \u06cc\u0627 \u0628\u0631\u0627\u0628\u0631 \u0628\u0627 \u062d\u062f \u0645\u062c\u0627\u0632 (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c 200).\n<\/li>\n<\/ul>\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> POST http:\/\/localhost:4000\/chat\/completions <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Content-Type: application\/json\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Authorization: Bearer sk-plugin-allowed-key\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-d<\/span> <span class=\"s1\">'{\n    \"model\": \"gpt-3.5-turbo\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Tell me a joke.\"}],\n    \"max_tokens\": 200\n  }'<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ol>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>*   **Expected Outcome:** The request should succeed. Logs should show \"Access Granted\" and \"Existing max_tokens (200) is within limit (512)\". The LLM call will use `max_tokens: 200`.\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<ol>\n<li>\n<p><strong>\u0645\u0648\u0631\u062f \u0622\u0632\u0645\u0627\u06cc\u0634 3: \u06a9\u0627\u0631\u0628\u0631 \u0631\u0627 \u0627\u0646\u06a9\u0627\u0631 \u06a9\u0631\u062f<\/strong><\/p>\n<ul>\n<li>    \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>sk-plugin-denied-key<\/code> \u06cc\u0627 <code>sk-no-metadata-key<\/code>\u0628\u0634\u0631\n<\/li>\n<\/ul>\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> POST http:\/\/localhost:4000\/chat\/completions <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Content-Type: application\/json\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Authorization: Bearer sk-plugin-denied-key\"<\/span> <span class=\"se\">\\<\/span>\n  <span class=\"nt\">-d<\/span> <span class=\"s1\">'{\n    \"model\": \"gpt-3.5-turbo\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Will this work?\"}]\n  }'<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ol>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>*   **Expected Outcome:** The request should **fail** with an **HTTP 403 Forbidden** error. The response body should contain the detail message: `\"Access Denied: Your API key does not have permission...\"`. The proxy logs should show \"Access Denied: Key metadata missing 'plugin_access: true'\". The LLM will *not* be called.\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0631\u0627 \u0648\u0627\u0631\u062f \u06a9\u0646\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_4_%D8%A7%D9%84%DA%AF%D9%88%D9%87%D8%A7%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D9%85%D8%B4%D8%AA%D8%B1%DA%A9_%D9%88_%D9%85%D9%88%D8%A7%D8%B1%D8%AF_%D8%A7%D8%B3%D8%AA%D9%81%D8%A7%D8%AF%D9%87\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 4: \u0627\u0644\u06af\u0648\u0647\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0645\u0634\u062a\u0631\u06a9 \u0648 \u0645\u0648\u0627\u0631\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u062f\u0631 <code>async_pre_call_hook<\/code> \u0637\u06cc\u0641 \u06af\u0633\u062a\u0631\u062f\u0647 \u0627\u06cc \u0627\u0632 \u0642\u0627\u0628\u0644\u06cc\u062a \u0647\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0631\u0627 \u0641\u0639\u0627\u0644 \u0645\u06cc \u06a9\u0646\u062f. \u062f\u0631 \u0627\u06cc\u0646\u062c\u0627 \u0628\u0631\u062e\u06cc \u0627\u0632 \u0627\u0644\u06af\u0648\u0647\u0627\u06cc \u0645\u0634\u062a\u0631\u06a9:<\/p>\n<p><strong>1. \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u0645\u062f\u0644 \u067e\u0648\u06cc\u0627:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u0645\u062f\u0644 LLM \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u060c \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u06a9\u0627\u0631\u0628\u0631 \u06cc\u0627 \u0645\u0646\u0637\u0642 \u062f\u06cc\u06af\u0631 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0628\u0627\u0632\u0631\u0633\u06cc \u06a9\u0631\u062f\u0646 <code>data<\/code> (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u0637\u0648\u0644 \u0633\u0631\u06cc\u0639 \u060c \u06a9\u0644\u0645\u0627\u062a \u06a9\u0644\u06cc\u062f\u06cc \u062e\u0627\u0635) \u06cc\u0627 <code>user_api_key_dict<\/code> (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u0631\u062f\u06cc\u0641 \u06a9\u0627\u0631\u0628\u0631 \u060c \u0628\u0648\u062f\u062c\u0647 \u0628\u0627\u0642\u06cc \u0645\u0627\u0646\u062f\u0647). \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0627\u062f\u0646 <code>data['model']<\/code> \u0642\u0628\u0644 \u0627\u0632 \u0628\u0627\u0632\u06af\u0634\u062a <code>None<\/code>\u0628\u0634\u0631<\/li>\n<li>\n<p><strong>\u0642\u0637\u0639\u0647 \u0645\u062b\u0627\u0644:<\/strong><\/p>\n<pre class=\"highlight python\"><code><span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"p\">...,<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[...]):<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">call_type<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">user_metadata<\/span> <span class=\"o\">=<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">metadata<\/span> <span class=\"ow\">or<\/span> <span class=\"p\">{}<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">user_metadata<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">tier<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">premium<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">'<\/span><span class=\"s\">model<\/span><span class=\"sh\">'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">gpt-4-turbo<\/span><span class=\"sh\">\"<\/span> <span class=\"c1\"># Route premium users to a better model\n<\/span>        <span class=\"k\">elif<\/span> <span class=\"nf\">len<\/span><span class=\"p\">(<\/span><span class=\"nf\">str<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">'<\/span><span class=\"s\">messages<\/span><span class=\"sh\">'<\/span><span class=\"p\">,<\/span> <span class=\"sh\">''<\/span><span class=\"p\">)))<\/span> <span class=\"o\">&gt;<\/span> <span class=\"mi\">4000<\/span><span class=\"p\">:<\/span> <span class=\"c1\"># Example: check prompt length\n<\/span>             <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">'<\/span><span class=\"s\">model<\/span><span class=\"sh\">'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">claude-3-haiku<\/span><span class=\"sh\">\"<\/span> <span class=\"c1\"># Route long prompts to a model with a large context\n<\/span>        <span class=\"c1\"># else: use default model passed in request\n<\/span>    <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ul>\n<p><strong>2. \u0627\u0639\u062a\u0628\u0627\u0631 \u0633\u0646\u062c\u06cc \u0648 \u0636\u062f \u0639\u0641\u0648\u0646\u06cc \u0648\u0631\u0648\u062f\u06cc:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u062d\u0627\u0635\u0644 \u06a9\u0646\u06cc\u062f \u06a9\u0647 Payload Payload \u0628\u0627 \u0645\u0639\u06cc\u0627\u0631\u0647\u0627\u06cc \u062e\u0627\u0635 (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632 \u060c \u0642\u0627\u0644\u0628 \u067e\u06cc\u0627\u0645 \u060c \u0645\u062d\u062a\u0648\u0627\u06cc \u0645\u062c\u0627\u0632) \u06cc\u0627 \u0636\u062f \u0639\u0641\u0648\u0646\u06cc \u06a9\u0631\u062f\u0646 \u0648\u0631\u0648\u062f\u06cc (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u0645\u0627\u0633\u06a9 PII) \u0645\u0637\u0627\u0628\u0642\u062a \u062f\u0627\u0631\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0628\u0627\u0632\u0631\u0633\u06cc \u06a9\u0631\u062f\u0646 <code>data<\/code>\u0628\u0634\u0631 \u062f\u0631 \u0635\u0648\u0631\u062a \u0639\u062f\u0645 \u0645\u0648\u0641\u0642\u06cc\u062a \u0627\u0639\u062a\u0628\u0627\u0631 \u060c \u0627\u0641\u0632\u0627\u06cc\u0634 \u062f\u0647\u06cc\u062f <code>HTTPException(status_code=400, detail=\"Validation failed...\")<\/code>\u0628\u0634\u0631 \u0628\u0631\u0627\u06cc \u0636\u062f \u0639\u0641\u0648\u0646\u06cc \u060c \u0627\u0635\u0644\u0627\u062d \u06a9\u0646\u06cc\u062f <code>data<\/code> (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c <code>data['messages']<\/code>) \u062f\u0631 \u062c\u0627\u06cc \u062e\u0648\u062f<\/li>\n<li>\n<p><strong>\u0645\u062b\u0627\u0644 \u0642\u0637\u0639\u0647 (\u0645\u0641\u0647\u0648\u0645 \u0646\u0642\u0627\u0628 \u0633\u0627\u0632\u06cc PII):<\/strong><\/p>\n<pre class=\"highlight python\"><code><span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"p\">...,<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[...]):<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">call_type<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span> <span class=\"ow\">and<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">messages<\/span><span class=\"sh\">\"<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">for<\/span> <span class=\"n\">message<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">'<\/span><span class=\"s\">messages<\/span><span class=\"sh\">'<\/span><span class=\"p\">]:<\/span>\n            <span class=\"k\">if<\/span> <span class=\"n\">message<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">'<\/span><span class=\"s\">role<\/span><span class=\"sh\">'<\/span><span class=\"p\">)<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">'<\/span><span class=\"s\">user<\/span><span class=\"sh\">'<\/span><span class=\"p\">:<\/span>\n                <span class=\"c1\"># Replace this with actual PII detection\/masking logic\n<\/span>                <span class=\"k\">if<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">email:<\/span><span class=\"sh\">\"<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">message<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">'<\/span><span class=\"s\">content<\/span><span class=\"sh\">'<\/span><span class=\"p\">,<\/span> <span class=\"sh\">''<\/span><span class=\"p\">):<\/span>\n                   <span class=\"c1\"># Caution: Simple string replacement is naive. Use proper libraries.\n<\/span>                   <span class=\"n\">message<\/span><span class=\"p\">[<\/span><span class=\"sh\">'<\/span><span class=\"s\">content<\/span><span class=\"sh\">'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">message<\/span><span class=\"p\">[<\/span><span class=\"sh\">'<\/span><span class=\"s\">content<\/span><span class=\"sh\">'<\/span><span class=\"p\">].<\/span><span class=\"nf\">replace<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">email:<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">masked_email:<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n                   <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Masked potential email in user message.<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span>\n<\/code><\/pre>\n<p><em>(\u062a\u0648\u062c\u0647: \u067e\u0648\u0634\u06cc\u062f\u0646 \u0645\u0646\u0627\u0633\u0628 PII \u0628\u0647 \u06a9\u062a\u0627\u0628\u062e\u0627\u0646\u0647 \u0647\u0627\u06cc \u0642\u0648\u06cc \u0645\u0627\u0646\u0646\u062f Presidio \u06cc\u0627 \u0645\u0646\u0637\u0642 \u0633\u0641\u0627\u0631\u0634\u06cc \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u062f).<\/em><\/p>\n<\/li>\n<\/ul>\n<p><strong>3. \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u063a\u0646\u06cc \u0633\u0627\u0632\u06cc:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u0628\u0647 \u0637\u0648\u0631 \u062e\u0648\u062f\u06a9\u0627\u0631 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0632\u0645\u06cc\u0646\u0647 \u06a9\u0627\u0631\u0628\u0631 \u06cc\u0627 \u062a\u0646\u0638\u06cc\u0645\u0627\u062a \u062c\u0647\u0627\u0646\u06cc \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0627\u0635\u0644\u0627\u062d <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0628\u0631\u0627\u06cc \u0627\u0636\u0627\u0641\u0647 \u06a9\u0631\u062f\u0646 \u06cc\u0627 \u0628\u0647 \u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc \u06a9\u0644\u06cc\u062f\u0647\u0627 \u0645\u0627\u0646\u0646\u062f <code>temperature<\/code>\u0628\u0627 <code>max_tokens<\/code>\u060c \u06cc\u0627 \u062a\u0632\u0631\u06cc\u0642 \u0633\u06cc\u0633\u062a\u0645 \u062e\u0627\u0635 \u06cc\u0627 \u0627\u0628\u0631\u062f\u0627\u062f\u0647.<\/li>\n<li>\n<p><strong>\u0642\u0637\u0639\u0647 \u0645\u062b\u0627\u0644:<\/strong><\/p>\n<pre class=\"highlight python\"><code><span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"p\">...,<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[...]):<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">call_type<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span>\n        <span class=\"c1\"># Ensure a user ID is always passed if available from the key\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">data<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">user<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"ow\">is<\/span> <span class=\"bp\">None<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">user_id<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">\"<\/span><span class=\"s\">user<\/span><span class=\"sh\">\"<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">user_id<\/span>\n\n        <span class=\"c1\"># Add default safety settings if not provided\n<\/span>        <span class=\"k\">if<\/span> <span class=\"n\">data<\/span><span class=\"p\">.<\/span><span class=\"nf\">get<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">safety_settings<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"ow\">is<\/span> <span class=\"bp\">None<\/span><span class=\"p\">:<\/span>\n             <span class=\"n\">data<\/span><span class=\"p\">[<\/span><span class=\"sh\">\"<\/span><span class=\"s\">safety_settings<\/span><span class=\"sh\">\"<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span><span class=\"sh\">\"<\/span><span class=\"s\">block_hate_speech<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span> <span class=\"bp\">True<\/span><span class=\"p\">}<\/span> <span class=\"c1\"># Example\n<\/span>    <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ul>\n<p><strong>4. \u0627\u062c\u0631\u0627\u06cc \u0633\u06cc\u0627\u0633\u062a (\u0628\u0648\u062f\u062c\u0647 \u060c \u0645\u062c\u0648\u0632\u0647\u0627):<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc\u06cc \u0631\u0627 \u06a9\u0647 \u0646\u0642\u0636 \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0647\u0627\u06cc \u0628\u0648\u062f\u062c\u0647 \u06cc\u0627 \u0642\u0648\u0627\u0646\u06cc\u0646 \u0645\u062c\u0648\u0632 \u062a\u0639\u0631\u06cc\u0641 \u0634\u062f\u0647 \u062f\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u06a9\u0644\u06cc\u062f\/\u062a\u06cc\u0645\/\u06a9\u0627\u0631\u0628\u0631 \u0627\u0633\u062a \u060c \u0631\u062f \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0628\u0648\u062f\u062c\u0647\/\u0635\u0631\u0641 \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u06cc\u0627 \u067e\u0631\u0686\u0645 \u0647\u0627\u06cc \u0645\u062c\u0648\u0632 \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0627\u0632 \u0622\u0646 \u0628\u062e\u0648\u0627\u0646\u06cc\u062f <code>user_api_key_dict<\/code>\u0628\u0634\u0631 \u0627\u06af\u0631 \u0633\u06cc\u0627\u0633\u062a\u06cc \u0646\u0642\u0636 \u0634\u062f\u0647 \u0627\u0633\u062a \u060c \u0645\u0637\u0631\u062d \u06a9\u0646\u06cc\u062f <code>HTTPException<\/code> (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c 403 \u0645\u0645\u0646\u0648\u0639 \u0628\u0631\u0627\u06cc \u0645\u062c\u0648\u0632\u0647\u0627 \u060c 429 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u06cc\u0634 \u0627\u0632 \u062d\u062f \u0628\u0631\u0627\u06cc \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0628\u0648\u062f\u062c\u0647\/\u0646\u0631\u062e).<\/li>\n<li>\n<p><strong>\u0645\u062b\u0627\u0644 \u0642\u0637\u0639\u0647 (\u0628\u0631\u0631\u0633\u06cc \u0628\u0648\u062f\u062c\u0647 \u0633\u0627\u062f\u0647):<\/strong><\/p>\n<pre class=\"highlight python\"><code><span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"p\">...,<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[...]):<\/span>\n    <span class=\"n\">key_spend<\/span> <span class=\"o\">=<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">spend<\/span> <span class=\"ow\">or<\/span> <span class=\"mf\">0.0<\/span>\n    <span class=\"n\">key_max_budget<\/span> <span class=\"o\">=<\/span> <span class=\"n\">user_api_key_dict<\/span><span class=\"p\">.<\/span><span class=\"n\">max_budget<\/span>\n\n    <span class=\"k\">if<\/span> <span class=\"n\">key_max_budget<\/span> <span class=\"ow\">is<\/span> <span class=\"ow\">not<\/span> <span class=\"bp\">None<\/span> <span class=\"ow\">and<\/span> <span class=\"n\">key_spend<\/span> <span class=\"o\">&gt;=<\/span> <span class=\"n\">key_max_budget<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">raise<\/span> <span class=\"nc\">HTTPException<\/span><span class=\"p\">(<\/span>\n            <span class=\"n\">status_code<\/span><span class=\"o\">=<\/span><span class=\"mi\">429<\/span><span class=\"p\">,<\/span> <span class=\"c1\"># Use 429 for budget\/rate limits\n<\/span>            <span class=\"n\">detail<\/span><span class=\"o\">=<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">API Key budget limit exceeded. Spend: <\/span><span class=\"si\">{<\/span><span class=\"n\">key_spend<\/span><span class=\"si\">}<\/span><span class=\"s\">, Budget: <\/span><span class=\"si\">{<\/span><span class=\"n\">key_max_budget<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span>\n        <span class=\"p\">)<\/span>\n    <span class=\"c1\"># Add checks for team budget, user budget etc. if needed\n<\/span>    <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span>\n<\/code><\/pre>\n<p><em>(\u062a\u0648\u062c\u0647: \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0646\u0631\u062e \u062f\u0627\u062e\u0644\u06cc \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u0628\u0631\u0627\u06cc \u0628\u0631\u0631\u0633\u06cc \u0647\u0627\u06cc \u0632\u0645\u0627\u0646 \u0648\u0627\u0642\u0639\u06cc \u060c \u067e\u06cc\u0634\u0631\u0641\u062a\u0647 \u062a\u0631 \u0627\u0633\u062a. \u0627\u06cc\u0646 \u06cc\u06a9 \u067e\u06cc\u0634 \u0628\u0631\u0631\u0633\u06cc \u0633\u0627\u062f\u0647 \u0627\u0633\u062a).<\/em><\/p>\n<\/li>\n<\/ul>\n<p><strong>5. \u0645\u0646\u0637\u0642 \u0631\u062f \u0633\u0641\u0627\u0631\u0634\u06cc:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0645\u062d\u0631\u06a9 \u0645\u062d\u062a\u0648\u0627\u06cc \u062e\u0627\u0635 \u06cc\u0627 \u0642\u0648\u0627\u0646\u06cc\u0646 \u062a\u062c\u0627\u0631\u06cc \u0631\u062f \u06a9\u0646\u06cc\u062f \u060c \u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u06cc\u06a9 \u067e\u06cc\u0627\u0645 \u0633\u0641\u0627\u0631\u0634\u06cc \u0631\u0627 \u0628\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0628\u0627\u0632\u0631\u0633\u06cc \u06a9\u0631\u062f\u0646 <code>data<\/code>\u0628\u0634\u0631 \u062f\u0631 \u0635\u0648\u0631\u062a \u0631\u0639\u0627\u06cc\u062a \u0645\u0639\u06cc\u0627\u0631\u0647\u0627\u06cc \u0631\u062f:\n<ul>\n<li>    \u0628\u0631\u0627\u06cc \u0686\u062a\/\u062a\u06a9\u0645\u06cc\u0644: <code>return \"Your request was rejected because...\"<\/code>\n<\/li>\n<li>    \u0628\u0631\u0627\u06cc \u0627\u0646\u0648\u0627\u0639 \u062f\u06cc\u06af\u0631 \u06cc\u0627 \u062e\u0637\u0627\u0647\u0627\u06cc \u062e\u0627\u0635 \u062a\u0631: <code>raise HTTPException(...)<\/code>\n<\/li>\n<\/ul>\n<\/li>\n<li>\n<p><strong>\u0642\u0637\u0639\u0647 \u0645\u062b\u0627\u0644 (\u0628\u0627\u0632\u06af\u0634\u062a \u0631\u0634\u062a\u0647):<\/strong><\/p>\n<pre class=\"highlight python\"><code><span class=\"kn\">from<\/span> <span class=\"n\">litellm.utils<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">get_formatted_prompt<\/span> <span class=\"c1\"># Helper to get text\n<\/span>\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">async_pre_call_hook<\/span><span class=\"p\">(<\/span><span class=\"n\">self<\/span><span class=\"p\">,<\/span> <span class=\"p\">...,<\/span> <span class=\"n\">data<\/span><span class=\"p\">:<\/span> <span class=\"nb\">dict<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"p\">:<\/span> <span class=\"n\">Literal<\/span><span class=\"p\">[...]):<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">call_type<\/span> <span class=\"o\">==<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">completion<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span>\n        <span class=\"k\">try<\/span><span class=\"p\">:<\/span>\n            <span class=\"n\">prompt_text<\/span> <span class=\"o\">=<\/span> <span class=\"nf\">get_formatted_prompt<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"o\">=<\/span><span class=\"n\">data<\/span><span class=\"p\">,<\/span> <span class=\"n\">call_type<\/span><span class=\"o\">=<\/span><span class=\"n\">call_type<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">if<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">forbidden phrase<\/span><span class=\"sh\">\"<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">prompt_text<\/span><span class=\"p\">.<\/span><span class=\"nf\">lower<\/span><span class=\"p\">():<\/span>\n                <span class=\"k\">return<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">Request rejected due to containing a forbidden phrase.<\/span><span class=\"sh\">\"<\/span> <span class=\"c1\"># LiteLLM formats this\n<\/span>        <span class=\"k\">except<\/span> <span class=\"nb\">Exception<\/span> <span class=\"k\">as<\/span> <span class=\"n\">e<\/span><span class=\"p\">:<\/span>\n             <span class=\"n\">litellm<\/span><span class=\"p\">.<\/span><span class=\"nf\">print_verbose<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Error getting formatted prompt: <\/span><span class=\"si\">{<\/span><span class=\"n\">e<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">return<\/span> <span class=\"bp\">None<\/span>\n<\/code><\/pre>\n<\/li>\n<\/ul>\n<p><strong>6. \u062a\u062e\u0645\u06cc\u0646 \u0648 \u067e\u06cc\u0634\u06af\u06cc\u0631\u06cc \u0647\u0632\u06cc\u0646\u0647:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u0647\u0632\u06cc\u0646\u0647 \u0628\u0627\u0644\u0642\u0648\u0647 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0631\u0627 \u062a\u062e\u0645\u06cc\u0646 \u0628\u0632\u0646\u06cc\u062f (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u0628\u0631 \u0627\u0633\u0627\u0633 \u0646\u0634\u0627\u0646\u0647 \u0647\u0627\u06cc \u0633\u0631\u06cc\u0639 \u0628\u0631\u0627\u06cc \u062a\u06a9\u0645\u06cc\u0644) \u0648 \u062f\u0631 \u0635\u0648\u0631\u062a \u0641\u0631\u0627\u062a\u0631 \u0627\u0632 \u0622\u0633\u062a\u0627\u0646\u0647 \u062a\u0639\u0631\u06cc\u0641 \u0634\u062f\u0647 \u062f\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u060c \u0622\u0646 \u0631\u0627 \u0631\u062f \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>litellm.token_counter<\/code> (\u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0645\u062a\u0646 async \u06cc\u0627 \u0641\u0631\u0627\u062e\u0648\u0627\u0646\u06cc \u0647\u0645\u0632\u0645\u0627\u0646 \u0628\u0627 \u062f\u0642\u062a \u0633\u0627\u0632\u06af\u0627\u0631 \u0645\u06cc \u06a9\u0646\u062f) <code>data['messages']<\/code>\u0628\u0634\u0631 \u0645\u062d\u0627\u0633\u0628\u0647 \u0647\u0632\u06cc\u0646\u0647 \u062a\u062e\u0645\u06cc\u0646 \u0632\u062f\u0647 \u0634\u062f\u0647 \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 <code>litellm.model_cost<\/code>\u0628\u0634\u0631 \u0645\u0642\u0627\u06cc\u0633\u0647 \u062f\u0631 \u0628\u0631\u0627\u0628\u0631 a <code>max_request_cost<\/code> \u0645\u0642\u062f\u0627\u0631 \u0627\u0632 <code>user_api_key_dict.metadata<\/code>\u0628\u0634\u0631 \u0628\u0627\u0644\u0627 \u0628\u0631\u062f\u0646 <code>HTTPException(400)<\/code> \u0627\u06af\u0631 \u062e\u06cc\u0644\u06cc \u0632\u06cc\u0627\u062f \u0628\u0627\u0634\u062f<\/li>\n<li>  <strong>\u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u062f:<\/strong> \u0627\u062c\u0631\u0627\u06cc \u062f\u0642\u06cc\u0642 \u0628\u0631\u0627\u06cc \u062c\u0644\u0648\u06af\u06cc\u0631\u06cc \u0627\u0632 \u0627\u0641\u0632\u0648\u062f\u0646 \u062a\u0623\u062e\u06cc\u0631 \u0628\u06cc\u0634 \u0627\u0632 \u062d\u062f. \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u0628\u0631\u0627\u06cc \u0633\u0646\u0627\u0631\u06cc\u0648\u0647\u0627\u06cc \u0628\u0633\u06cc\u0627\u0631 \u067e\u0631 \u0647\u0632\u06cc\u0646\u0647 \u0628\u0647\u062a\u0631 \u0628\u0627\u0634\u062f.<\/li>\n<\/ul>\n<p><strong>7. \u062a\u0646\u0638\u06cc\u0645 \u0622\u0632\u0645\u0627\u06cc\u0634 A\/B:<\/strong><\/p>\n<ul>\n<li>  <strong>\u0647\u062f\u0641:<\/strong> \u062f\u0631\u0635\u062f\u06cc \u0627\u0632 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627 \u0628\u0631\u0627\u06cc \u06cc\u06a9 \u0645\u062f\u0644 \u062e\u0627\u0635 \u0628\u0647 \u06cc\u06a9 \u0645\u062f\u0644 \u062c\u0627\u06cc\u06af\u0632\u06cc\u0646 \u06cc\u0627 \u0627\u0636\u0627\u0641\u0647 \u06a9\u0631\u062f\u0646 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u062e\u0627\u0635 \u0628\u0631\u0627\u06cc \u0631\u062f\u06cc\u0627\u0628\u06cc \u062a\u0633\u062a \u0647\u0627\u06cc A\/B \u0631\u0627 \u0645\u0633\u06cc\u0631 \u062f\u0647\u06cc\u062f.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642:<\/strong> \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>random.random()<\/code> \u06cc\u0627 \u0634\u0646\u0627\u0633\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0628\u0631\u0627\u06cc \u062a\u0639\u06cc\u06cc\u0646 \u0627\u06cc\u0646\u06a9\u0647 \u0622\u06cc\u0627 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631 \u06af\u0631\u0648\u0647 \u0622\u0632\u0645\u0648\u0646 \u0642\u0631\u0627\u0631 \u0645\u06cc \u06af\u06cc\u0631\u062f \u06cc\u0627 \u062e\u06cc\u0631. \u062a\u063a\u06cc\u06cc\u0631 \u062f\u0627\u062f\u0646 <code>data['model']<\/code> \u06cc\u0627 \u067e\u0631\u0686\u0645 \u0647\u0627 \u0631\u0627 \u0628\u0647 <code>data['metadata']<\/code> (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c <code>data['metadata']['ab_test_group'] = 'B'<\/code>). \u0627\u0632 <code>cache<\/code> \u0627\u06af\u0631 \u0628\u0631\u0627\u06cc \u06cc\u06a9 \u062c\u0644\u0633\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0628\u0647 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u0686\u0633\u0628\u0646\u062f\u0647 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u062f.<\/li>\n<\/ul>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_5_%D8%AA%D8%B9%D8%A7%D9%85%D9%84_%D8%A8%D8%A7_%D9%82%D9%84%D8%A7%D8%A8_%D9%87%D8%A7%DB%8C_%D8%AF%DB%8C%DA%AF%D8%B1_%D9%87%D9%85%D8%A7%D9%87%D9%86%DA%AF%DB%8C_%DA%A9%D9%84%DB%8C%D8%AF%DB%8C_%D8%A7%D8%B3%D8%AA\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 5: \u062a\u0639\u0627\u0645\u0644 \u0628\u0627 \u0642\u0644\u0627\u0628 \u0647\u0627\u06cc \u062f\u06cc\u06af\u0631 (\u0647\u0645\u0627\u0647\u0646\u06af\u06cc \u06a9\u0644\u06cc\u062f\u06cc \u0627\u0633\u062a)<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u062f\u0631 \u062d\u0627\u0644\u06cc \u06a9\u0647 <code>async_pre_call_hook<\/code> \u0642\u062f\u0631\u062a\u0645\u0646\u062f \u0627\u0633\u062a \u060c \u0627\u063a\u0644\u0628 \u0628\u0627 \u0647\u0645\u0627\u0647\u0646\u06af\u06cc \u0633\u0627\u06cc\u0631 \u0642\u0644\u0627\u0628 \u0647\u0627 \u060c \u0628\u0647 \u0648\u06cc\u0698\u0647 \u0628\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc \u062d\u0627\u0644\u062a \u060c \u0628\u0647\u062a\u0631\u06cc\u0646 \u06a9\u0627\u0631 \u0631\u0627 \u0627\u0646\u062c\u0627\u0645 \u0645\u06cc \u062f\u0647\u062f:<\/p>\n<ul>\n<li>  <strong><code>async_log_success_event<\/code>  \/ <code>async_log_failure_event<\/code>:<\/strong> \u0636\u0631\u0648\u0631\u06cc \u0628\u0631\u0627\u06cc <em>\u0628\u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc<\/em> \u062f\u0648\u0644\u062a \u0628\u0631 \u0627\u0633\u0627\u0633 <em>\u0646\u062a\u06cc\u062c\u0647<\/em> \u0627\u0632 \u062a\u0645\u0627\u0633 LLM \u062f\u0631 <code>parallel_request_limiter<\/code> \u0627\u0632 \u0627\u06cc\u0646\u0647\u0627 \u0628\u0631\u0627\u06cc \u06a9\u0627\u0647\u0634 \u062a\u0639\u062f\u0627\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0648 \u0628\u0647 \u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc TPM \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u062f <em>\u067e\u0633 \u0627\u0632<\/em> \u062a\u0645\u0627\u0633 \u062a\u0645\u0627\u0645 \u0645\u06cc \u0634\u0648\u062f. \u0627\u06af\u0631 \u0642\u0644\u0627\u0628 \u0642\u0628\u0644 \u0627\u0632 \u062a\u0645\u0627\u0633 \u0634\u0645\u0627 \u0628\u0647 \u0637\u0631\u0632 \u062e\u0648\u0634 \u0628\u06cc\u0646 \u067e\u06cc\u0634\u062e\u0648\u0627\u0646 \u0631\u0627 \u0627\u0641\u0632\u0627\u06cc\u0634 \u0645\u06cc \u062f\u0647\u062f \u06cc\u0627 \u067e\u0631\u0686\u0645 \u0631\u0627 \u062a\u0646\u0638\u06cc\u0645 \u0645\u06cc \u06a9\u0646\u062f \u060c \u0627\u062d\u062a\u0645\u0627\u0644\u0627\u064b \u0628\u0647 \u0627\u06cc\u0646 \u0642\u0644\u0627\u0628 \u0647\u0627\u06cc \u067e\u0633 \u0627\u0632 \u062a\u0645\u0627\u0633 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u062f \u062a\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0645\u0648\u0641\u0642\u06cc\u062a \u06cc\u0627 \u0639\u062f\u0645 \u0645\u0648\u0641\u0642\u06cc\u062a \u0622\u0646 \u062d\u0627\u0644\u062a \u0631\u0627 \u062a\u0646\u0638\u06cc\u0645 \u06cc\u0627 \u067e\u0627\u06a9 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong><code>async_post_call_success_hook<\/code>:<\/strong> \u0627\u06af\u0631 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0627\u0635\u0644\u0627\u062d \u062f\u0627\u0631\u062f \u0645\u0641\u06cc\u062f \u0627\u0633\u062a <em>\u067e\u0627\u0633\u062e<\/em> \u0628\u0631 \u0627\u0633\u0627\u0633 \u0627\u0642\u062f\u0627\u0645\u0627\u062a \u0627\u0646\u062c\u0627\u0645 \u0634\u062f\u0647 \u062f\u0631 \u0642\u0644\u0627\u0628 \u0642\u0628\u0644 \u0627\u0632 \u062a\u0645\u0627\u0633 \u06cc\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u062d\u0627\u0644\u062a \u0646\u0647\u0627\u06cc\u06cc. \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0646\u0631\u062e \u0627\u0632 \u0622\u0646 \u0628\u0631\u0627\u06cc \u0627\u0641\u0632\u0648\u062f\u0646 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u062f <code>X-RateLimit<\/code> \u0647\u062f\u0631\u0647 \u0647\u0627\u06cc\u06cc \u06a9\u0647 \u062a\u0639\u062f\u0627\u062f \u0646\u0647\u0627\u06cc\u06cc \u0631\u0627 \u0645\u0646\u0639\u06a9\u0633 \u0645\u06cc \u06a9\u0646\u0646\u062f.<\/li>\n<li>  <strong><code>async_moderation_hook<\/code>:<\/strong> \u062f\u0648\u06cc\u062f\u0646 <em>\u0628\u0647 \u0637\u0648\u0631 \u0645\u0648\u0627\u0632\u06cc<\/em> \u0628\u0647 \u062a\u0645\u0627\u0633 \u0627\u0635\u0644\u06cc LLM. \u0627\u06af\u0631 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0634\u0627\u0645\u0644 \u06cc\u06a9 \u0686\u06a9 \u0628\u0627\u0644\u0642\u0648\u0647 \u06a9\u0646\u062f (\u0645\u0627\u0646\u0646\u062f \u0627\u0639\u062a\u062f\u0627\u0644 \u0645\u062d\u062a\u0648\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647) \u0627\u0633\u062a \u06a9\u0647 \u0646\u0628\u0627\u06cc\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u0635\u0644\u06cc \u0631\u0627 \u0645\u0633\u062f\u0648\u062f \u06a9\u0646\u062f \u060c \u0628\u0647 \u062c\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0627\u06cc\u0646 \u0642\u0644\u0627\u0628 \u0631\u0627 \u062f\u0631 \u0646\u0638\u0631 \u0628\u06af\u06cc\u0631\u06cc\u062f <code>async_pre_call_hook<\/code>\u0628\u0634\u0631 \u0622\u06af\u0627\u0647 \u0628\u0627\u0634\u06cc\u062f \u06a9\u0647 \u0627\u06af\u0631 <code>async_moderation_hook<\/code> \u06cc\u06a9 \u0627\u0633\u062a\u062b\u0646\u0627 \u0631\u0627 \u0627\u06cc\u062c\u0627\u062f \u0645\u06cc \u06a9\u0646\u062f \u060c \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u067e\u0627\u0633\u062e \u0648\u0627\u0642\u0639\u06cc LLM \u0631\u0627 \u0646\u0627\u062f\u06cc\u062f\u0647 \u0628\u06af\u06cc\u0631\u062f.<\/li>\n<\/ul>\n<p><strong>\u0645\u062b\u0627\u0644 \u0647\u0645\u0627\u0647\u0646\u06af\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u062d\u0627\u0644\u062a (\u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0646\u0631\u062e \u0645\u0641\u0647\u0648\u0645\u06cc):<\/strong><\/p>\n<ol>\n<li> <strong><code>async_pre_call_hook<\/code>:<\/strong>\n<ul>\n<li>    \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0647\u0627 \u0631\u0627 \u0627\u0632 <code>user_api_key_dict<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u062a\u0639\u062f\u0627\u062f \u0641\u0639\u0644\u06cc \u0631\u0627 \u0627\u0632 \u0622\u0646 \u0628\u062e\u0648\u0627\u0646\u06cc\u062f <code>cache<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u0627\u06af\u0631 <code>count + 1 &gt; limit<\/code>\u060c \u0628\u0627\u0644\u0627 \u0628\u0631\u062f\u0646 <code>HTTPException(429)<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u062f\u0631 \u0635\u0648\u0631\u062a \u0645\u062c\u0627\u0632 \u060c <code>await cache.async_increment_cache(key, 1)<\/code>\u0628\u0634\u0631<\/li>\n<li>    \u0628\u0627\u0632\u06af\u0634\u062a <code>None<\/code>\u0628\u0634\u0631<\/li>\n<\/ul>\n<\/li>\n<li> <strong><code>async_log_success_event<\/code>:<\/strong>\n<ul>\n<li>  <code>await cache.async_increment_cache(key, -1)<\/code>  (\u062a\u0639\u062f\u0627\u062f \u06a9\u0627\u0647\u0634).<\/li>\n<li>    TPM \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0628\u0647 \u0631\u0648\u0632 \u06a9\u0646\u06cc\u062f <code>response_obj.usage<\/code>\u0628\u0634\u0631<\/li>\n<\/ul>\n<\/li>\n<li> <strong><code>async_log_failure_event<\/code>:<\/strong>\n<ul>\n<li>    \u0627\u06af\u0631 <code>exception<\/code> \u0627\u0633\u062a \u060c <em>\u0646\u0647<\/em> 429 \u0645\u0637\u0631\u062d \u0634\u062f\u0647 \u062a\u0648\u0633\u0637 \u0642\u0644\u0627\u0628 \u0642\u0628\u0644 \u0627\u0632 \u062a\u0645\u0627\u0633:\n<ul>\n<li>  <code>await cache.async_increment_cache(key, -1)<\/code>  (\u062a\u0639\u062f\u0627\u062f \u06a9\u0627\u0647\u0634).<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_6_%D8%AA%DA%A9%D9%86%DB%8C%DA%A9_%D9%87%D8%A7%DB%8C_%D9%BE%DB%8C%D8%B4%D8%B1%D9%81%D8%AA%D9%87\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 6: \u062a\u06a9\u0646\u06cc\u06a9 \u0647\u0627\u06cc \u067e\u06cc\u0634\u0631\u0641\u062a\u0647<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li>  <strong>\u062d\u0627\u0644\u062a \u067e\u06cc\u0686\u06cc\u062f\u0647 \u0628\u0627 <code>cache<\/code>:<\/strong> \u0628\u06cc\u0634 \u0627\u0632 \u067e\u06cc\u0634\u062e\u0648\u0627\u0646 \u0647\u0627\u06cc \u0633\u0627\u062f\u0647 \u0631\u0627 \u0630\u062e\u06cc\u0631\u0647 \u06a9\u0646\u06cc\u062f. \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u06cc\u0627 \u0631\u0634\u062a\u0647 \u0647\u0627\u06cc JSON \u06a9\u0647 \u062c\u0644\u0633\u0627\u062a \u06a9\u0627\u0631\u0628\u0631 \u060c \u062d\u0627\u0644\u062a \u0647\u0627\u06cc \u0641\u0631\u0622\u06cc\u0646\u062f \u0686\u0646\u062f \u0645\u0631\u062d\u0644\u0647 \u0627\u06cc \u06cc\u0627 \u0632\u0645\u0627\u0646 \u0628\u0646\u062f\u06cc \u0641\u0639\u0627\u0644\u06cc\u062a \u0627\u062e\u06cc\u0631 \u0631\u0627 \u0646\u0634\u0627\u0646 \u0645\u06cc \u062f\u0647\u062f. \u0628\u0647 \u06cc\u0627\u062f \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f \u06a9\u0647 \u0627\u0632 TTL \u0647\u0627\u06cc \u0645\u0646\u0627\u0633\u0628 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>\u062a\u0645\u0627\u0633 \u0647\u0627\u06cc API \u062e\u0627\u0631\u062c\u06cc:<\/strong> \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 <em>\u0642\u0648\u0637\u06cc<\/em> <code>await<\/code>  \u062a\u0645\u0627\u0633 \u0628\u0627 \u062e\u062f\u0645\u0627\u062a \u062e\u0627\u0631\u062c\u06cc (\u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u0645\u062b\u0627\u0644 \u060c \u06cc\u06a9 \u0646\u0642\u0637\u0647 \u062a\u0635\u0645\u06cc\u0645 \u06af\u06cc\u0631\u06cc \u0633\u06cc\u0627\u0633\u062a \u060c \u06cc\u06a9 \u0633\u0631\u0648\u06cc\u0633 \u067e\u0631\u0686\u0645 \u0648\u06cc\u0698\u06af\u06cc \u060c API \u062a\u0634\u062e\u06cc\u0635 PII). <strong>\u0627\u0632 \u0627\u062d\u062a\u06cc\u0627\u0637 \u0634\u062f\u06cc\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f:<\/strong>\n<ul>\n<li>    \u0627\u06cc\u0646 \u062a\u0645\u0627\u0633 \u0647\u0627 \u0631\u0627 \u0633\u0631\u06cc\u0639 \u0648 \u0642\u0627\u0628\u0644 \u0627\u0639\u062a\u0645\u0627\u062f \u0646\u06af\u0647 \u062f\u0627\u0631\u06cc\u062f. \u06cc\u06a9 \u062a\u0645\u0627\u0633 \u062e\u0627\u0631\u062c\u06cc \u0622\u0647\u0633\u062a\u0647 \u062a\u0623\u062e\u06cc\u0631 \u0631\u0627 \u0628\u0647 <em>\u0647\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a<\/em>\u0628\u0634\u0631<\/li>\n<li>    \u0628\u0631\u0627\u06cc \u062a\u0645\u0627\u0633 \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u060c \u0631\u0633\u06cc\u062f\u06af\u06cc \u0628\u0647 \u062e\u0637\u0627\u06cc \u0642\u0648\u06cc \u0648 \u0632\u0645\u0627\u0646 \u0628\u0646\u062f\u06cc \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u062f. \u0627\u062c\u0627\u0632\u0647 \u0646\u062f\u0647\u06cc\u062f \u06a9\u0647 \u06cc\u06a9 \u0633\u0631\u0648\u06cc\u0633 \u062e\u0627\u0631\u062c\u06cc \u0646\u0627\u06a9\u0627\u0645 \u060c \u067e\u0631\u0648\u06a9\u0633\u06cc \u0634\u0645\u0627 \u0631\u0627 \u067e\u0627\u06cc\u06cc\u0646 \u0628\u06cc\u0627\u0648\u0631\u062f.<\/li>\n<li>    \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u062a\u0645\u0627\u0633 \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u0646\u062a\u0627\u06cc\u062c \u0630\u062e\u06cc\u0631\u0647 \u0631\u0627 \u062f\u0631 \u0646\u0638\u0631 \u0628\u06af\u06cc\u0631\u06cc\u062f <code>DualCache<\/code>\u0628\u0634\u0631<\/li>\n<\/ul>\n<\/li>\n<li>  <strong>\u062f\u0633\u062a\u06a9\u0627\u0631\u06cc \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647:<\/strong> \u0628\u0631\u0627\u06cc \u06a9\u0627\u0631\u0647\u0627\u06cc\u06cc \u0645\u0627\u0646\u0646\u062f \u0642\u0627\u0644\u0628 \u0628\u0646\u062f\u06cc \u0633\u0631\u06cc\u0639 \u0648 \u06cc\u0627 \u062a\u0628\u062f\u06cc\u0644 \u062f\u0627\u062f\u0647 \u0647\u0627 \u060c \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u062a\u0648\u0627\u0628\u0639 \u06cc\u0627 \u06a9\u0644\u0627\u0633 \u0647\u0627\u06cc \u06cc\u0627\u0631\u06cc \u0631\u0627 \u062a\u0639\u0631\u06cc\u0641 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u062f\u0631 \u062c\u0627\u06cc \u062f\u06cc\u06af\u0631\u06cc \u062f\u0631 \u067e\u0631\u0648\u0646\u062f\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f \u06cc\u0627 \u0645\u0627\u0698\u0648\u0644 \u0647\u0627\u06cc \u0648\u0627\u0631\u062f\u0627\u062a\u06cc \u062a\u0639\u0631\u06cc\u0641 \u0634\u062f\u0647 \u0627\u0646\u062f. \u0645\u0646\u0637\u0642 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f \u0631\u0627 \u062a\u0645\u06cc\u0632 \u0648 \u0645\u062a\u0645\u0631\u06a9\u0632 \u0628\u0631 \u0631\u0648\u06cc \u0627\u0631\u06a9\u0633\u062a\u0631\u0627\u0633\u06cc\u0648\u0646 \u0646\u06af\u0647 \u062f\u0627\u0631\u06cc\u062f.<\/li>\n<\/ul>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_7_%D8%A8%D9%87%D8%AA%D8%B1%DB%8C%D9%86_%D8%B4%DB%8C%D9%88%D9%87_%D9%87%D8%A7_%D9%88_%D9%85%D8%B4%DA%A9%D9%84%D8%A7%D8%AA\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 7: \u0628\u0647\u062a\u0631\u06cc\u0646 \u0634\u06cc\u0648\u0647 \u0647\u0627 \u0648 \u0645\u0634\u06a9\u0644\u0627\u062a<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<ul>\n<li>  <strong>\u0639\u0645\u0644\u06a9\u0631\u062f \u0645\u0647\u0645 \u0627\u0633\u062a:<\/strong> \u06a9\u062f \u062f\u0631 \u062f\u0627\u062e\u0644 <code>async_pre_call_hook<\/code> \u062a\u0623\u062e\u06cc\u0631 \u0631\u0627 \u0627\u0636\u0627\u0641\u0647 \u0645\u06cc \u06a9\u0646\u062f <em>\u067e\u06cc\u0634 \u0627\u0632<\/em> \u062a\u0645\u0627\u0633 LLM \u0622\u0646 \u0631\u0627 \u0644\u0627\u063a\u0631 \u0648 \u0633\u0631\u06cc\u0639 \u0646\u06af\u0647 \u062f\u0627\u0631\u06cc\u062f. \u0627\u0632 I\/O \u0647\u0645\u0632\u0645\u0627\u0646 \u060c \u0645\u062d\u0627\u0633\u0628\u0627\u062a \u067e\u06cc\u0686\u06cc\u062f\u0647 \u06cc\u0627 \u062d\u0644\u0642\u0647 \u0647\u0627\u06cc \u0646\u0627\u06a9\u0627\u0631\u0622\u0645\u062f \u062e\u0648\u062f\u062f\u0627\u0631\u06cc \u06a9\u0646\u06cc\u062f. \u062f\u0631 \u0635\u0648\u0631\u062a \u0644\u0632\u0648\u0645 \u067e\u0631\u0648\u0641\u0627\u06cc\u0644. \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>asyncio.create_task<\/code> \u0628\u0631\u0627\u06cc \u0628\u0647 \u0631\u0648\u0632\u0631\u0633\u0627\u0646\u06cc \u0647\u0627\u06cc \u067e\u0633 \u0632\u0645\u06cc\u0646\u0647 \u063a\u06cc\u0631 \u0628\u062d\u0631\u0627\u0646\u06cc (\u0645\u0627\u0646\u0646\u062f \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u062f\u0631 \u0645\u062d\u062f\u0648\u062f \u06a9\u0646\u0646\u062f\u0647 \u0646\u0631\u062e).<\/li>\n<li>  <strong>\u0631\u0633\u06cc\u062f\u06af\u06cc \u0628\u0647 \u062e\u0637\u0627\u06cc \u0642\u0648\u06cc:<\/strong> \u0645\u0646\u0637\u0642 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f \u0631\u0627 \u062f\u0631 <code>try...except<\/code>\u0628\u0634\u0631 \u06cc\u06a9 \u0627\u0633\u062a\u062b\u0646\u0627\u0621 \u063a\u06cc\u0631\u0642\u0627\u0646\u0648\u0646\u06cc \u062f\u0631 \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 \u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u062f \u0645\u0633\u062f\u0648\u062f \u06a9\u0646\u062f <em>\u0647\u0645\u0647<\/em> \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0639\u0628\u0648\u0631 \u0627\u0632 \u067e\u0631\u0648\u06a9\u0633\u06cc. \u062e\u0637\u0627\u0647\u0627\u06cc \u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 \u0628\u0631\u0627\u06cc \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0628\u0647 \u0648\u0636\u0648\u062d \u062f\u0631 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f (<code>litellm.print_verbose<\/code> \u06cc\u0627 \u06cc\u06a9 \u0627\u0644\u06af\u0648\u06cc \u0645\u0646\u0627\u0633\u0628). \u062f\u0631 \u0646\u0638\u0631 \u0628\u06af\u06cc\u0631\u06cc\u062f \u06a9\u0647 \u0627\u06af\u0631 \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 \u0628\u0627 \u06cc\u06a9 \u062e\u0637\u0627\u06cc \u063a\u06cc\u0631 \u0645\u0646\u062a\u0638\u0631\u0647 \u0631\u0648\u0628\u0631\u0648 \u0634\u0648\u062f \u0686\u0647 \u0627\u062a\u0641\u0627\u0642\u06cc \u0645\u06cc \u0627\u0641\u062a\u062f &#8211; \u062f\u0631 \u0635\u0648\u0631\u062a \u0631\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a (\u0627\u0641\u0632\u0627\u06cc\u0634 <code>HTTPException(500)<\/code>) \u06cc\u0627 \u0627\u062c\u0627\u0632\u0647 \u062f\u0647\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0627\u062f\u0627\u0645\u0647 \u062f\u0647\u062f (\u0634\u06a9\u0633\u062a)\u061f<\/li>\n<li>  <strong>\u0622\u06af\u0627\u0647\u06cc \u0627\u0645\u0646\u06cc\u062a\u06cc:<\/strong> \u0647\u0646\u06af\u0627\u0645 \u0627\u0635\u0644\u0627\u062d <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u060c \u0628\u0647 \u0648\u06cc\u0698\u0647 <code>data['messages']<\/code> \u06cc\u0627 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc\u06cc \u06a9\u0647 \u0628\u0631 \u0631\u0641\u062a\u0627\u0631 \u0645\u062f\u0644 \u062a\u0623\u062b\u06cc\u0631 \u0645\u06cc \u06af\u0630\u0627\u0631\u062f. \u0647\u0631 \u0648\u0631\u0648\u062f\u06cc \u062e\u0627\u0631\u062c\u06cc \u0631\u0627 \u06a9\u0647 \u062a\u0648\u0633\u0637 \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u0634\u0648\u062f \u060c \u0636\u062f\u0639\u0641\u0648\u0646\u06cc \u06a9\u0646\u06cc\u062f. \u0627\u0632 \u0645\u0639\u0631\u0641\u06cc \u0622\u0633\u06cc\u0628 \u067e\u0630\u06cc\u0631\u06cc \u0647\u0627\u06cc \u0633\u0631\u06cc\u0639 \u062a\u0632\u0631\u06cc\u0642 \u062e\u0648\u062f\u062f\u0627\u0631\u06cc \u06a9\u0646\u06cc\u062f.<\/li>\n<li>  <strong>idempotency:<\/strong> \u062f\u0631 \u062d\u0627\u0644\u06cc \u06a9\u0647 \u0628\u0647 \u0637\u0648\u0631 \u0645\u0639\u0645\u0648\u0644 \u06cc\u06a9 \u0628\u0627\u0631 \u062f\u0631 \u0647\u0631 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u06a9\u0646\u06cc\u062f \u060c \u062f\u0631 \u0646\u0638\u0631 \u0628\u06af\u06cc\u0631\u06cc\u062f \u06a9\u0647 \u0622\u06cc\u0627 \u062a\u0639\u0627\u0645\u0644 \u0628\u0627 \u062d\u0627\u0641\u0638\u0647 \u0646\u0647\u0627\u0646 \u06cc\u0627 \u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06cc \u062e\u0627\u0631\u062c\u06cc \u0646\u06cc\u0627\u0632 \u0628\u0647 idempotent \u062f\u0627\u0631\u062f (\u0628\u0631\u0627\u06cc \u0627\u062c\u0631\u0627\u06cc \u0686\u0646\u062f\u06cc\u0646 \u0628\u0627\u0631 \u0628\u0627 \u0647\u0645\u0627\u0646 \u0646\u062a\u06cc\u062c\u0647 \u0627\u06cc\u0645\u0646 \u0627\u0633\u062a) \u0627\u06af\u0631 \u062a\u0631\u0645\u06cc\u0645 \u0647\u0627 \u0628\u0647 \u0637\u0648\u0631 \u0628\u0627\u0644\u0642\u0648\u0647 \u0645\u06cc \u062a\u0648\u0627\u0646\u0646\u062f \u0642\u0644\u0627\u0628 \u0631\u0627 \u062f\u0648\u0628\u0627\u0631\u0647 \u062a\u062d\u0631\u06cc\u06a9 \u06a9\u0646\u0646\u062f (\u0647\u0631\u0686\u0646\u062f \u06a9\u0647 \u0645\u0646\u0637\u0642 \u0622\u0632\u0645\u0627\u06cc\u0634 \u0645\u062c\u062f\u062f Litellm \u0627\u062a\u0641\u0627\u0642 \u0645\u06cc \u0627\u0641\u062a\u062f <em>\u067e\u0633 \u0627\u0632<\/em> \u0627\u062c\u0631\u0627\u06cc \u0627\u0648\u0644\u06cc\u0647 \u0642\u0644\u0627\u0628 \u0634\u06a9\u0633\u062a \u0645\u06cc \u062e\u0648\u0631\u062f).<\/li>\n<li>  <strong>\u0627\u0635\u0644\u0627\u062d\u0627\u062a \u067e\u0627\u06a9:<\/strong> \u0622\u0646 \u0631\u0627 \u0622\u0634\u06a9\u0627\u0631 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0628\u0647 \u0686\u0647 \u06a9\u0627\u0631\u06cc \u0627\u0646\u062c\u0627\u0645 \u0645\u06cc \u062f\u0647\u062f <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0646\u0638\u0631\u0627\u062a \u0631\u0627 \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u062f. \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u062d\u0627\u0635\u0644 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a \u062f\u0631 \u06cc\u06a9 \u0628\u0627\u0631 \u0645\u0639\u062a\u0628\u0631 \u0628\u0631\u0627\u06cc LLM Target \u0627\u0633\u062a.<\/li>\n<li>  <strong>\u0645\u0646\u0637\u0642 \u0647\u062f\u0641\u0645\u0646\u062f (<code>call_type<\/code>):<\/strong> \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>call_type<\/code> \u0628\u0631\u0631\u0633\u06cc \u0647\u0627 \u0628\u0631\u0627\u06cc \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u0627\u0632 \u0627\u06cc\u0646\u06a9\u0647 \u0627\u0641\u0632\u0648\u0646\u0647 \u0634\u0645\u0627 \u0641\u0642\u0637 \u0628\u0631 \u0639\u0645\u0644\u06cc\u0627\u062a LLM \u062f\u0631 \u0646\u0638\u0631 \u06af\u0631\u0641\u062a\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a.<\/li>\n<li>  <strong>\u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc:<\/strong> \u0627\u0632 \u0645\u0642\u0627\u062f\u06cc\u0631 \u0633\u062e\u062a \u06a9\u062f\u06af\u0630\u0627\u0631\u06cc \u062e\u0648\u062f\u062f\u0627\u0631\u06cc \u06a9\u0646\u06cc\u062f. \u0627\u0632 <code>__init__<\/code> \u0631\u0648\u0634 \u06a9\u0644\u0627\u0633 \u06a9\u0646\u062a\u0631\u0644 \u06a9\u0646\u0646\u062f\u0647 \u062e\u0648\u062f \u0628\u0631\u0627\u06cc \u067e\u0630\u06cc\u0631\u0634 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc (\u0645\u0627\u0646\u0646\u062f <code>max_tokens_limit<\/code> \u062f\u0631 \u0645\u062b\u0627\u0644 \u0645\u0627) \u06cc\u0627 \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f \u0631\u0627 \u0628\u0631\u0627\u06cc \u062e\u0648\u0627\u0646\u062f\u0646 \u062a\u0646\u0638\u06cc\u0645\u0627\u062a \u0627\u0632 <code>user_api_key_dict.metadata<\/code>\u0628\u0634\u0631<\/li>\n<li>  <strong>\u0622\u0632\u0645\u0627\u06cc\u0634:<\/strong> \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f \u0631\u0627 \u0628\u0627 \u0648\u0631\u0648\u062f\u06cc \u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u060c \u0645\u0648\u0627\u0631\u062f \u0644\u0628\u0647 \u060c \u06a9\u0644\u06cc\u062f\u0647\u0627\u06cc \u0645\u0639\u062a\u0628\u0631 \u060c \u06a9\u0644\u06cc\u062f\u0647\u0627\u06cc \u0646\u0627\u0645\u0639\u062a\u0628\u0631 \u060c \u0645\u062a\u0641\u0627\u0648\u062a \u062a\u0633\u062a \u06a9\u0646\u06cc\u062f <code>call_type<\/code>\u0633 \u060c \u0648 \u0633\u0646\u0627\u0631\u06cc\u0648\u0647\u0627\u06cc \u0634\u06a9\u0633\u062a \u0627\u0646\u062a\u0638\u0627\u0631 \u0645\u06cc \u0631\u0648\u062f.<\/li>\n<\/ul>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%82%D8%B3%D9%85%D8%AA_8_%D8%A7%D8%B4%DA%A9%D8%A7%D9%84_%D8%B2%D8%AF%D8%A7%DB%8C%DB%8C_%D8%A7%D9%81%D8%B2%D9%88%D9%86%D9%87_%D8%AE%D9%88%D8%AF\"><\/span>\n<p>  \u0642\u0633\u0645\u062a 8: \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u0642\u0644\u0627\u0628 \u0647\u0627\u06cc \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0645\u06cc \u062a\u0648\u0627\u0646\u0646\u062f \u0645\u0634\u06a9\u0644 \u0628\u0627\u0634\u0646\u062f. \u062f\u0631 \u0627\u06cc\u0646\u062c\u0627 \u0686\u0646\u062f \u0627\u0633\u062a\u0631\u0627\u062a\u0698\u06cc \u0622\u0648\u0631\u062f\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a:<\/p>\n<ol>\n<li> <strong>\u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 Verbose:<\/strong> \u0641\u0639\u0627\u0644 \u06a9\u0631\u062f\u0646 <code>litellm_settings.set_verbose: true<\/code> \u062f\u0631 \u0634\u0645\u0627 <code>config.yaml<\/code>\u0628\u0634\u0631 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0631\u062f\u0646 <code>litellm.print_verbose(...)<\/code> \u0628\u0631\u0627\u06cc \u0686\u0627\u067e \u0645\u0642\u0627\u062f\u06cc\u0631 \u0645\u062a\u063a\u06cc\u0631 \u060c \u0645\u0631\u0627\u062d\u0644 \u0627\u062c\u0631\u0627\u06cc \u0648 \u062a\u0635\u0645\u06cc\u0645\u0627\u062a \u0628\u0647 \u0637\u0648\u0631 \u0622\u0632\u0627\u062f\u0627\u0646\u0647 \u062f\u0631 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f.<\/li>\n<li> <strong>\u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f:<\/strong> \u0628\u0631\u0627\u06cc \u0648\u0631\u0648\u062f \u0628\u0647 \u0633\u06cc\u0633\u062a\u0645 \u0633\u0627\u062e\u062a\u0627\u0631\u06cc \u0628\u06cc\u0634\u062a\u0631 \u060c \u06cc\u06a9 \u0644\u0627\u062c\u06af\u0631 Python \u0645\u0646\u0627\u0633\u0628 \u0631\u0627 \u062f\u0631 \u06a9\u0644\u0627\u0633 \u0627\u0641\u0632\u0648\u0646\u0647 \u062e\u0648\u062f \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u06a9\u0646\u06cc\u062f.<\/li>\n<li> <strong>\u0639\u0628\u0627\u0631\u0627\u062a \u0686\u0627\u067e:<\/strong> \u067e\u06cc\u0631 \u062e\u0648\u0628 <code>print()<\/code> \u0628\u06cc\u0627\u0646\u06cc\u0647 \u0647\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u0646\u062f \u062f\u0631 \u0637\u0648\u0644 \u062a\u0648\u0633\u0639\u0647 \u0627\u0648\u0644\u06cc\u0647 \u0645\u0641\u06cc\u062f \u0628\u0627\u0634\u0646\u062f (\u0628\u0647 \u06cc\u0627\u062f \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f \u06a9\u0647 \u0628\u0639\u062f\u0627\u064b \u0622\u0646\u0647\u0627 \u0631\u0627 \u062d\u0630\u0641 \u06a9\u0646\u06cc\u062f).<\/li>\n<li> <strong>\u0642\u0644\u0627\u0628 \u0631\u0627 \u062c\u062f\u0627 \u06a9\u0646\u06cc\u062f:<\/strong> \u0628\u0647 \u0637\u0648\u0631 \u0645\u0648\u0642\u062a \u062f\u0631 \u0645\u0648\u0631\u062f \u0633\u0627\u06cc\u0631 \u062a\u0645\u0627\u0633\u0647\u0627\u06cc \u0628\u0631\u06af\u0634\u062a\u06cc \u06cc\u0627 \u0645\u0646\u0637\u0642 \u067e\u06cc\u0686\u06cc\u062f\u0647 \u062f\u0631 \u0627\u0638\u0647\u0627\u0631 \u0646\u0638\u0631 \u06a9\u0646\u06cc\u062f <code>config.yaml<\/code> \u0628\u0631\u0627\u06cc \u062c\u062f\u0627\u0633\u0627\u0632\u06cc \u0631\u0641\u062a\u0627\u0631 \u062e\u0627\u0635 \u062e\u0648\u062f <code>async_pre_call_hook<\/code>\u0628\u0634\u0631<\/li>\n<li> <strong>\u0648\u0631\u0648\u062f\u06cc \u0647\u0627\u06cc \u062e\u0627\u0635 \u0622\u0632\u0645\u0648\u0646:<\/strong> \u06a9\u0627\u0631\u062f\u0633\u062a\u06cc <code>curl<\/code> \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u06cc\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0645\u0634\u062a\u0631\u06cc \u0647\u0627\u06cc API (\u0645\u0627\u0646\u0646\u062f \u067e\u0633\u062a\u0686\u06cc \u060c \u0628\u06cc \u062e\u0648\u0627\u0628\u06cc) \u0628\u0631\u0627\u06cc \u0627\u0631\u0633\u0627\u0644 \u0628\u0627\u0631\u0647\u0627\u06cc \u062e\u0627\u0635 \u06a9\u0647 \u0645\u0633\u06cc\u0631\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0631\u0627 \u062f\u0631 \u0645\u0646\u0637\u0642 \u0642\u0644\u0627\u0628 \u0634\u0645\u0627 \u0627\u06cc\u062c\u0627\u062f \u0645\u06cc \u06a9\u0646\u062f.<\/li>\n<li> <strong>\u0628\u0631\u0631\u0633\u06cc \u06a9\u0631\u062f\u0646 <code>UserAPIKeyAuth<\/code>:<\/strong> \u06a9\u0644 \u0631\u0627 \u0686\u0627\u067e \u06a9\u0646\u06cc\u062f <code>user_api_key_dict<\/code> \u062f\u0631 \u0634\u0631\u0648\u0639 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f \u062f\u0631 \u0647\u0646\u06af\u0627\u0645 \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0628\u0631\u0627\u06cc \u062a\u0623\u06cc\u06cc\u062f \u0627\u06cc\u0646\u06a9\u0647 \u0632\u0645\u06cc\u0646\u0647 \u060c \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0647\u0627 \u0648 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0645\u0648\u0631\u062f \u0627\u0646\u062a\u0638\u0627\u0631 \u0631\u0627 \u062f\u0631\u06cc\u0627\u0641\u062a \u0645\u06cc \u06a9\u0646\u06cc\u062f.<\/li>\n<li> <strong>\u0628\u0631\u0631\u0633\u06cc \u06a9\u0631\u062f\u0646 <code>data<\/code>:<\/strong> \u0686\u0627\u067e <code>data<\/code> \u0641\u0631\u0647\u0646\u06af \u0644\u063a\u062a \u0642\u0628\u0644 \u0648 \u0628\u0639\u062f \u0627\u0632 \u0627\u0635\u0644\u0627\u062d\u0627\u062a \u0628\u0631\u0627\u06cc \u062a\u0623\u06cc\u06cc\u062f \u062a\u063a\u06cc\u06cc\u0631\u0627\u062a.<\/li>\n<li> <strong>Debugger Python:<\/strong> \u0627\u06af\u0631 \u0628\u0647 \u0635\u0648\u0631\u062a \u0645\u062d\u0644\u06cc \u0627\u062c\u0631\u0627 \u0645\u06cc \u0634\u0648\u062f \u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f <code>pdb<\/code> \u06cc\u0627 \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc IDE \u0634\u0645\u0627. \u06cc\u06a9 \u0646\u0642\u0637\u0647 \u0634\u06a9\u0633\u062a \u0631\u0627 \u062a\u0646\u0638\u06cc\u0645 \u06a9\u0646\u06cc\u062f (<code>import pdb; pdb.set_trace()<\/code>) \u062f\u0627\u062e\u0644 \u0631\u0648\u0634 \u0642\u0644\u0627\u0628 \u062e\u0648\u062f. \u0634\u0645\u0627 \u0628\u0627\u06cc\u062f \u0627\u0634\u06a9\u0627\u0644 \u0632\u062f\u0627\u06cc\u06cc \u0631\u0627 \u0628\u0647 \u062f\u0648\u06cc\u062f\u0646 \u0648\u0635\u0644 \u06a9\u0646\u06cc\u062f <code>litellm<\/code> \u0631\u0648\u0646\u062f<\/li>\n<\/ol>\n<hr\/>\n<h3><span class=\"ez-toc-section\" id=\"%D9%86%D8%AA%DB%8C%D8%AC%D9%87_%DA%AF%DB%8C%D8%B1%DB%8C_%D8%B3%D8%A7%D8%AE%D8%AA_%D8%8C_%DA%AF%D8%B3%D8%AA%D8%B1%D8%B4_%D8%8C_%DA%A9%D9%86%D8%AA%D8%B1%D9%84\"><\/span>\n<p>  \u0646\u062a\u06cc\u062c\u0647 \u06af\u06cc\u0631\u06cc: \u0633\u0627\u062e\u062a \u060c \u06af\u0633\u062a\u0631\u0634 \u060c \u06a9\u0646\u062a\u0631\u0644<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u062f\u0631 <code>async_pre_call_hook<\/code> \u0633\u0646\u06af \u0628\u0646\u0627\u06cc \u067e\u0644\u0627\u06af\u06cc\u0646 \u0647\u0627\u06cc \u062a\u0623\u062b\u06cc\u0631\u06af\u0630\u0627\u0631 \u0633\u0627\u062e\u062a\u0645\u0627\u0646 \u0628\u0631\u0627\u06cc \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0627\u0633\u062a. \u0627\u06cc\u0646 \u067e\u0631\u0648\u06a9\u0633\u06cc \u0631\u0627 \u0627\u0632 \u06cc\u06a9 \u0627\u0631\u0633\u0627\u0644 \u06a9\u0646\u0646\u062f\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0633\u0627\u062f\u0647 \u0628\u0647 \u06cc\u06a9 \u0644\u0627\u06cc\u0647 \u0628\u0633\u06cc\u0627\u0631 \u0642\u0627\u0628\u0644 \u062a\u0646\u0638\u06cc\u0645 \u0648 \u0647\u0648\u0634\u0645\u0646\u062f \u062a\u0628\u062f\u06cc\u0644 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0642\u0627\u062f\u0631 \u0628\u0647 \u0627\u062c\u0631\u0627\u06cc \u062e\u0637 \u0645\u0634\u06cc \u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647 \u060c \u062a\u0637\u0628\u06cc\u0642 \u067e\u0648\u06cc\u0627 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627 \u0648 \u0627\u062f\u063a\u0627\u0645 \u0645\u0646\u0637\u0642 \u062f\u0633\u062a \u0648 \u067e\u0646\u062c\u0647 \u0646\u0631\u0645 \u0645\u0633\u062a\u0642\u06cc\u0645 \u062f\u0631 \u06af\u0631\u062f\u0634 \u06a9\u0627\u0631 LLM \u0627\u0633\u062a.<\/p>\n<p>\u0628\u0627 \u062a\u0633\u0644\u0637 \u0628\u0631 \u0627\u0645\u0636\u0627\u06cc \u0622\u0646 \u060c \u062f\u0631\u06a9 \u062f\u0627\u062f\u0647 \u0647\u0627\u06cc \u0645\u062a\u0646\u06cc \u06a9\u0647 \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u062f\u0647\u062f (<code>UserAPIKeyAuth<\/code>\u0628\u0627 <code>cache<\/code>\u060c \u063a\u0646\u06cc \u0634\u062f\u0647 <code>data<\/code>) \u060c \u0648 \u0628\u0627 \u062f\u0642\u062a \u06a9\u0646\u062a\u0631\u0644 \u062e\u0631\u0648\u062c\u06cc \u0622\u0646 \u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0641\u0632\u0648\u0646\u0647 \u0647\u0627\u06cc\u06cc \u0628\u0633\u0627\u0632\u06cc\u062f \u06a9\u0647 \u0627\u0645\u0646\u06cc\u062a \u0631\u0627 \u0627\u0641\u0632\u0627\u06cc\u0634 \u0645\u06cc \u062f\u0647\u062f \u060c \u0647\u0632\u06cc\u0646\u0647 \u0647\u0627 \u0631\u0627 \u0628\u0647 \u0637\u0648\u0631 \u0645\u0624\u062b\u0631 \u0645\u062f\u06cc\u0631\u06cc\u062a \u0645\u06cc \u06a9\u0646\u062f \u060c \u062a\u062c\u0631\u0628\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0631\u0627 \u0628\u0647\u0628\u0648\u062f \u0645\u06cc \u0628\u062e\u0634\u062f \u0648 \u067e\u0631\u0648\u06a9\u0633\u06cc LITELLM \u0631\u0627 \u062f\u0642\u06cc\u0642\u0627\u064b \u0645\u0637\u0627\u0628\u0642 \u0628\u0627 \u0646\u06cc\u0627\u0632\u0647\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0634\u0645\u0627 \u062a\u0646\u0638\u06cc\u0645 \u0645\u06cc \u06a9\u0646\u062f.<\/p>\n<p>\u0633\u0627\u062f\u0647 \u0631\u0627 \u0634\u0631\u0648\u0639 \u06a9\u0646\u06cc\u062f \u060c \u06a9\u0627\u0645\u0644\u0627\u064b \u0622\u0632\u0645\u0627\u06cc\u0634 \u06a9\u0646\u06cc\u062f \u060c \u0628\u0647 \u0628\u0647\u062a\u0631\u06cc\u0646 \u0631\u0648\u0634\u0647\u0627 \u0631\u0639\u0627\u06cc\u062a \u06a9\u0646\u06cc\u062f \u0648 \u0627\u0632 \u0642\u062f\u0631\u062a \u06a9\u0627\u0645\u0644 \u0627\u06cc\u0646 \u0642\u0644\u0627\u0628 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f \u062a\u0627 \u0633\u0637\u062d \u0628\u0639\u062f\u06cc \u0633\u0641\u0627\u0631\u0634\u06cc \u0633\u0627\u0632\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc \u062f\u0631\u0648\u0627\u0632\u0647 AI \u062e\u0648\u062f \u0628\u0627\u0632 \u06a9\u0646\u06cc\u062f. \u0648\u0635\u0644 \u06a9\u0631\u062f\u0646 \u0645\u0628\u0627\u0631\u06a9!<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u0634\u0645\u0627 \u0627\u0632 \u067e\u0631\u0648\u06a9\u0633\u06cc Litellm \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u062f \u0648 \u0627\u0632 \u062a\u0648\u0627\u0646\u0627\u06cc\u06cc \u0622\u0646 \u062f\u0631 \u0645\u062a\u062d\u062f \u06a9\u0631\u062f\u0646 \u062f\u0633\u062a\u0631\u0633\u06cc \u0628\u0647 \u0645\u062f\u0644\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0632\u0628\u0627\u0646 \u0628\u0632\u0631\u06af (LLM) \u0642\u062f\u0631\u062f\u0627\u0646\u06cc \u0645\u06cc \u06a9\u0646\u06cc\u062f. \u0627\u0645\u0627 \u0627\u06af\u0631 \u0628\u0647 \u0628\u06cc\u0634\u062a\u0631 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f \u0686\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u062f\u061f \u0686\u0647 \u0645\u06cc \u0634\u0648\u062f \u0627\u06af\u0631 \u0634\u0645\u0627 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u062f \u06a9\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc \u067e\u0648\u06cc\u0627 \u0631\u0627 \u0628\u0631 \u0627\u0633\u0627\u0633 \u0628\u0648\u062f\u062c\u0647 \u06a9\u0627\u0631\u0628\u0631 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u06a9\u0646\u06cc\u062f \u060c \u0633\u06cc\u0627\u0633\u062a \u0647\u0627\u06cc &hellip;<\/p>\n","protected":false},"author":2,"featured_media":103761,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[339],"tags":[],"class_list":["post-103760","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dev"],"_links":{"self":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/103760","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/comments?post=103760"}],"version-history":[{"count":0,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/103760\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media\/103761"}],"wp:attachment":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media?parent=103760"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/categories?post=103760"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/tags?post=103760"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}