{"id":85412,"date":"2024-11-25T18:20:12","date_gmt":"2024-11-25T14:50:12","guid":{"rendered":"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/"},"modified":"2024-11-25T18:20:12","modified_gmt":"2024-11-25T14:50:12","slug":"llama-32-on-lambda-k9p","status":"publish","type":"post","link":"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/","title":{"rendered":"Llama 3.2 \u062f\u0631 Lambda &#8211; DEV Community"},"content":{"rendered":"<p>Summarize this content to 400 words in Persian Lang<br \/>\n              Llama 3.2 1B \u06cc\u06a9 \u0645\u062f\u0644 \u0647\u0648\u0634 \u0645\u0635\u0646\u0648\u0639\u06cc \u0633\u0628\u06a9 \u0648\u0632\u0646 \u0627\u0633\u062a \u06a9\u0647 \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u062c\u0627\u0644\u0628 \u0645\u06cc \u06a9\u0646\u062f \u0632\u06cc\u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646 \u0622\u0646 \u0631\u0627 \u0646\u0633\u0628\u062a\u0627\u064b \u0633\u0631\u06cc\u0639 \u0628\u062f\u0648\u0646 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0634\u062a\u0627\u0628 GPU \u0627\u062c\u0631\u0627 \u06a9\u0631\u062f.<\/p>\n<p>\u0645\u0627 \u0627\u0632 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc\u06cc \u0627\u0632 Hugging Face \u0648 Nitric \u0628\u0631\u0627\u06cc \u0646\u0634\u0627\u0646 \u062f\u0627\u062f\u0646 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0622\u0646 \u0648 \u0645\u062f\u06cc\u0631\u06cc\u062a \u0632\u06cc\u0631\u0633\u0627\u062e\u062a\u200c\u0647\u0627\u06cc \u0627\u0637\u0631\u0627\u0641\u060c \u0645\u0627\u0646\u0646\u062f \u0645\u0633\u06cc\u0631\u0647\u0627\u06cc API \u0648 \u0627\u0633\u062a\u0642\u0631\u0627\u0631\u060c \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n<p>\u067e\u06cc\u0634 \u0646\u06cc\u0627\u0632\u0647\u0627<\/p>\n<p>  \u0631\u0627\u0647 \u0627\u0646\u062f\u0627\u0632\u06cc \u067e\u0631\u0648\u0698\u0647<\/p>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0628\u0627 \u0627\u06cc\u062c\u0627\u062f \u06cc\u06a9 \u067e\u0631\u0648\u0698\u0647 \u062c\u062f\u06cc\u062f \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0642\u0627\u0644\u0628 \u0634\u0631\u0648\u0639 \u067e\u0627\u06cc\u062a\u0648\u0646 Nitric \u0634\u0631\u0648\u0639 \u06a9\u0646\u06cc\u0645.<\/p>\n<p>nitric new llama py-starter<br \/>\ncd llama<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>\u062f\u0631 \u0645\u0631\u062d\u0644\u0647 \u0628\u0639\u062f\u060c \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u067e\u0627\u06cc\u0647 \u0631\u0627 \u0646\u0635\u0628 \u06a9\u0646\u06cc\u0645\u060c \u0633\u067e\u0633 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u0627\u0636\u0627\u0641\u06cc \u0631\u0627 \u06a9\u0647 \u0628\u0647 \u0637\u0648\u0631 \u062e\u0627\u0635 \u0628\u0631\u0627\u06cc \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u0645\u062f\u0644 \u0632\u0628\u0627\u0646 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u0645\u060c \u0627\u0636\u0627\u0641\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u0645.<\/p>\n<p># Install the base dependencies<br \/>\nuv sync<br \/>\n# Add the llama-cpp-python dependency<br \/>\nuv add llama-cpp-python<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>  \u0645\u062f\u0644 \u0644\u0627\u0645\u0627 \u0631\u0627 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f<\/p>\n<p>Llama 3.2 \u062f\u0631 \u0627\u0646\u062f\u0627\u0632\u0647\u200c\u0647\u0627 \u0648 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc\u200c\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0645\u0648\u062c\u0648\u062f \u0627\u0633\u062a \u06a9\u0647 \u0647\u0631 \u06a9\u062f\u0627\u0645 \u0627\u0632 \u0646\u0638\u0631 \u0639\u0645\u0644\u06a9\u0631\u062f\u060c \u062f\u0642\u062a \u0648 \u0645\u0646\u0627\u0628\u0639 \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632\u060c \u0645\u0639\u0627\u0648\u0636\u0647\u200c\u0647\u0627\u06cc \u062e\u0627\u0635 \u062e\u0648\u062f \u0631\u0627 \u062f\u0627\u0631\u0646\u062f. \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u0628\u062f\u0648\u0646 \u0634\u062a\u0627\u0628 GPU\u060c \u0645\u0627\u0646\u0646\u062f AWS Lambda\u060c \u0645\u0647\u0645 \u0627\u0633\u062a \u06a9\u0647 \u0645\u062f\u0644\u06cc \u0631\u0627 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0633\u0628\u06a9 \u0648 \u06a9\u0627\u0631\u0622\u0645\u062f \u0628\u0627\u0634\u062f \u062a\u0627 \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u062d\u0627\u0635\u0644 \u0634\u0648\u062f \u06a9\u0647 \u062f\u0631 \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0647\u0627\u06cc \u0622\u0646 \u0645\u062d\u06cc\u0637 \u0627\u062c\u0631\u0627 \u0645\u06cc \u0634\u0648\u062f.<\/p>\n<p>\u0645\u0627 \u0627\u0632 \u06cc\u06a9 \u0646\u0633\u062e\u0647 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0647 \u0634\u062f\u0647 \u0627\u0632 \u0645\u062f\u0644 \u0633\u0628\u06a9 \u0648\u0632\u0646 Llama 1B\u060c \u0628\u0647 \u0648\u06cc\u0698\u0647 Llama-3.2-1B-Instruct-Q4_K_M.gguf \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0628\u0627 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0627\u0633\u06cc\u0648\u0646 \u0622\u0634\u0646\u0627\u06cc\u06cc \u0646\u062f\u0627\u0631\u06cc\u062f\u060c \u0627\u06cc\u0646 \u062a\u06a9\u0646\u06cc\u06a9\u06cc \u0627\u0633\u062a \u06a9\u0647 \u0627\u0646\u062f\u0627\u0632\u0647 \u0648 \u0645\u0646\u0627\u0628\u0639 \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632 \u0645\u062f\u0644 \u0631\u0627 \u06a9\u0627\u0647\u0634 \u0645\u06cc\u200c\u062f\u0647\u062f\u060c \u06a9\u0647 \u062f\u0631 \u0645\u0648\u0631\u062f \u0645\u0627\u060c \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647\u200c\u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u0645\u0646\u0627\u0633\u0628 \u0645\u06cc\u200c\u06a9\u0646\u062f \u0627\u0645\u0627 \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u0628\u0631 \u062f\u0642\u062a \u0645\u062f\u0644 \u062a\u0623\u062b\u06cc\u0631 \u0628\u06af\u0630\u0627\u0631\u062f.<\/p>\n<p>\u062a\u06cc\u0645 LM Studio \u0686\u0646\u062f\u06cc\u0646 \u0646\u0633\u062e\u0647 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0647 \u0634\u062f\u0647 Llama 3.2 1B \u0631\u0627 \u062f\u0631 Hugging Face \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u06a9\u0646\u062f. \u0633\u0639\u06cc \u06a9\u0646\u06cc\u062f \u0646\u0633\u062e\u0647\u200c\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0631\u0627 \u0627\u0645\u062a\u062d\u0627\u0646 \u06a9\u0646\u06cc\u062f \u062a\u0627 \u0628\u0647\u062a\u0631\u06cc\u0646 \u0645\u0637\u0627\u0628\u0642 \u0628\u0627 \u0646\u06cc\u0627\u0632\u0647\u0627\u06cc \u0634\u0645\u0627 \u0631\u0627 \u0628\u06cc\u0627\u0628\u06cc\u062f\u060c \u0645\u0627\u0646\u0646\u062f Q5_K_M\u060c \u06a9\u0647 \u06a9\u0645\u06cc \u0628\u0632\u0631\u06af\u062a\u0631 \u0627\u0633\u062a \u0627\u0645\u0627 \u06a9\u06cc\u0641\u06cc\u062a \u0628\u0627\u0644\u0627\u062a\u0631\u06cc \u062f\u0627\u0631\u062f.<\/p>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0645\u062f\u0644 \u0627\u0646\u062a\u062e\u0627\u0628 \u0634\u062f\u0647 \u0631\u0627 \u062f\u0627\u0646\u0644\u0648\u062f \u06a9\u0631\u062f\u0647 \u0648 \u062f\u0631 \u067e\u0648\u0634\u0647 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc \u067e\u0631\u0648\u0698\u0647 \u062e\u0648\u062f \u0630\u062e\u06cc\u0631\u0647 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0644\u06cc\u0646\u06a9 \u062f\u0627\u0646\u0644\u0648\u062f Llama-3.2-1B-Instruct-Q4_K_M.gguf:<\/p>\n<p>mkdir models<br \/>\ncd models<br \/>\n# This model is 0.81GB, it may take a little while to download<br \/>\ncurl -OL https:\/\/huggingface.co\/bartowski\/Llama-3.2-1B-Instruct-GGUF\/resolve\/main\/Llama-3.2-1B-Instruct-Q4_K_M.gguf<br \/>\ncd ..<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>\u0633\u0627\u062e\u062a\u0627\u0631 \u067e\u0648\u0634\u0647 \u0634\u0645\u0627 \u0628\u0627\u06cc\u062f \u0628\u0647 \u0634\u06a9\u0644 \u0632\u06cc\u0631 \u0628\u0627\u0634\u062f:<\/p>\n<p>\/llama<br \/>\n    \/models<br \/>\n        Llama-3.2-1B-Instruct-Q4_K_M.gguf<br \/>\n    \/services<br \/>\n        api.py<br \/>\n    nitric.yaml<br \/>\n    pyproject.toml<br \/>\n    python.dockerfile<br \/>\n    python.dockerfile.ignore<br \/>\n    README.md<br \/>\n    uv.lock<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>  \u06cc\u06a9 \u0633\u0631\u0648\u06cc\u0633 \u0628\u0631\u0627\u06cc \u0627\u062c\u0631\u0627\u06cc \u0645\u062f\u0644 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f<\/p>\n<p>\u062f\u0631 \u0645\u0631\u062d\u0644\u0647 \u0628\u0639\u062f\u060c \u0627\u0632 Nitric \u0628\u0631\u0627\u06cc \u0627\u06cc\u062c\u0627\u062f \u06cc\u06a9 API HTTP \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u0645 \u06a9\u0647 \u0628\u0647 \u0634\u0645\u0627 \u0627\u0645\u06a9\u0627\u0646 \u0645\u06cc \u062f\u0647\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc\u06cc \u0631\u0627 \u0628\u0647 \u0645\u062f\u0644 Llama \u0627\u0631\u0633\u0627\u0644 \u06a9\u0646\u06cc\u062f \u0648 \u062e\u0631\u0648\u062c\u06cc \u0631\u0627 \u062f\u0631 \u06cc\u06a9 \u067e\u0627\u0633\u062e \u062f\u0631\u06cc\u0627\u0641\u062a \u06a9\u0646\u06cc\u062f. API \u062e\u0631\u0648\u062c\u06cc \u062e\u0627\u0645 \u0631\u0627 \u0627\u0632 \u0645\u062f\u0644 \u0628\u0631\u0645\u06cc \u06af\u0631\u062f\u0627\u0646\u062f\u060c \u0627\u0645\u0627 \u0634\u0645\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0647 \u062f\u0644\u062e\u0648\u0627\u0647 \u062a\u0646\u0638\u06cc\u0645 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0645\u062d\u062a\u0648\u06cc\u0627\u062a \u0631\u0627 \u062c\u0627\u06cc\u06af\u0632\u06cc\u0646 \u06a9\u0646\u06cc\u062f services\/api.py \u0628\u0627 \u06a9\u062f \u0632\u06cc\u0631 \u06a9\u0647 \u0645\u062f\u0644 Llama \u0631\u0627 \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u06a9\u0631\u062f\u0647 \u0648 \u0639\u0645\u0644\u06a9\u0631\u062f \u0633\u0631\u06cc\u0639 \u0631\u0627 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u06cc \u06a9\u0646\u062f. \u0628\u0631\u0627\u06cc \u062f\u0631\u06a9 \u06a9\u062f \u06a9\u0645\u06cc \u0632\u0645\u0627\u0646 \u0628\u06af\u0630\u0627\u0631\u06cc\u062f. \u06cc\u06a9 API \u0631\u0627 \u0628\u0627 \u06cc\u06a9 \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc \/ prompt \u062a\u0639\u0631\u06cc\u0641 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a POST \u0631\u0627 \u0628\u0627 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631 \u0628\u062f\u0646\u0647 \u0645\u06cc \u067e\u0630\u06cc\u0631\u062f. \u0627\u06cc\u0646 process_prompt \u062a\u0627\u0628\u0639 \u062f\u0633\u062a\u0648\u0631 \u0631\u0627 \u0628\u0647 \u0645\u062f\u0644 Llama \u0645\u06cc \u0641\u0631\u0633\u062a\u062f \u0648 \u067e\u0627\u0633\u062e \u0631\u0627 \u0628\u0631\u0645\u06cc \u06af\u0631\u062f\u0627\u0646\u062f.<\/p>\n<p>from nitric.resources import api<br \/>\nfrom nitric.application import Nitric<br \/>\nfrom nitric.context import HttpContext<br \/>\nfrom llama_cpp import Llama<\/p>\n<p># Load the locally stored Llama model<br \/>\nllm = Llama(model_path=&#8221;.\/models\/Llama-3.2-1B-Instruct-Q4_K_M.gguf&#8221;)<\/p>\n<p># Function to execute a prompt using the Llama model<br \/>\ndef process_prompt(user_prompt):<br \/>\n    system_prompt = &#8220;You are a helpful assistant.&#8221;<\/p>\n<p>    # See https:\/\/www.llama.com\/docs\/model-cards-and-prompt-formats\/llama3_1\/ for details about prompt format<br \/>\n    prompt = (<br \/>\n        # System Prompt<br \/>\n        f'&lt;|start_header_id|&gt;system&lt;|end_header_id|&gt;{system_prompt}&lt;|eot_id|&gt;&#8217;<br \/>\n        # User Prompt<br \/>\n        f'&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;{user_prompt}&lt;|eot_id|&gt;&#8217;<br \/>\n        # Start assistants turn (we leave this open ended as the assistant hasn&#8217;t started its turn)<br \/>\n        f'&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;&#8217;<br \/>\n    )<\/p>\n<p>    response = llm(<br \/>\n        prompt=prompt,<br \/>\n        # Unlimited, consider setting a token limit<br \/>\n        max_tokens=-1,<br \/>\n        temperature=0.7,<br \/>\n    )<\/p>\n<p>    return response<\/p>\n<p># Define an API for the prompt service<br \/>\nmain = api(&#8220;main&#8221;)<\/p>\n<p>@main.post(&#8220;\/prompt&#8221;)<br \/>\nasync def handle_prompt(ctx: HttpContext):<br \/>\n    # assume the input is text\/plain<br \/>\n    prompt = ctx.req.data<\/p>\n<p>    try:<br \/>\n        ctx.res.body = process_prompt(prompt)<br \/>\n    except Exception as e:<br \/>\n        print(f&#8221;Error processing prompt: {e}&#8221;)<br \/>\n        ctx.res.body = {&#8220;error&#8221;: str(e)}<br \/>\n        ctx.res.status = 500<\/p>\n<p>Nitric.run()<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>  \u062e\u0648\u0628\u060c \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0627\u06cc\u0646 \u06a9\u0627\u0631 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u0645!<\/p>\n<p>\u0627\u06a9\u0646\u0648\u0646 \u06a9\u0647 \u06cc\u06a9 API \u062a\u0639\u0631\u06cc\u0641 \u06a9\u0631\u062f\u0647 \u0627\u06cc\u062f\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u0622\u0646 \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u0645\u062d\u0644\u06cc \u0622\u0632\u0645\u0627\u06cc\u0634 \u06a9\u0646\u06cc\u0645. \u0627\u0644\u06af\u0648\u06cc \u0634\u0631\u0648\u0639 \u067e\u0627\u06cc\u062a\u0648\u0646 \u0627\u0632 python3.11-bookworm-slim \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u062a\u0635\u0648\u06cc\u0631 \u0645\u062d\u0641\u0638\u0647 \u0627\u0635\u0644\u06cc \u062e\u0648\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0645\u0646\u0627\u0633\u0628\u06cc \u0628\u0631\u0627\u06cc \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u0645\u062f\u0644 Llama \u0646\u062f\u0627\u0631\u062f. \u0627\u062c\u0627\u0632\u0647 \u062f\u0647\u06cc\u062f Dockerfile \u0631\u0627 \u0628\u0647 \u0631\u0648\u0632 \u06a9\u0646\u06cc\u0645 \u062a\u0627 \u0628\u0647 \u062c\u0627\u06cc \u0622\u0646 \u0627\u0632 Python3.11-bookworm (\u0646\u0633\u062e\u0647 \u063a\u06cc\u0631 \u0628\u0627\u0631\u06cc\u06a9) \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u0645.<\/p>\n<p>\u0628\u0647 \u0631\u0648\u0632 \u0631\u0633\u0627\u0646\u06cc \u062e\u0637 2:<\/p>\n<p>FROM python:3.11-bookworm1FROM python:3.11-bookworm<\/p>\n<p>\u0628\u0647 \u0631\u0648\u0632 \u0631\u0633\u0627\u0646\u06cc \u062e\u0637 19:<\/p>\n<p>FROM python:3.11-bookworm1FROM python:3.11-bookworm<\/p>\n<p>\u0627\u06a9\u0646\u0648\u0646 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u062e\u062f\u0645\u0627\u062a \u062e\u0648\u062f \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u0645\u062d\u0644\u06cc \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u0645:<\/p>\n<p>nitric run<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>nitric run  \u0628\u0631\u0646\u0627\u0645\u0647 \u0634\u0645\u0627 \u0631\u0627 \u062f\u0631 \u0638\u0631\u0641\u06cc \u0634\u0631\u0648\u0639 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0634\u0627\u0645\u0644 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 llama_cpp \u0627\u0633\u062a. \u0627\u06af\u0631 \u062a\u0631\u062c\u06cc\u062d \u0645\u06cc \u062f\u0647\u06cc\u062f \u0627\u0632 \u0646\u06cc\u062a\u0631\u06cc\u06a9 \u0627\u0633\u062a\u0627\u0631\u062a \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f\u060c \u0628\u0627\u06cc\u062f \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc llama-cpp-python \u0645\u0627\u0646\u0646\u062f CMake \u0648 LLVM \u0646\u0635\u0628 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u067e\u0633 \u0627\u0632 \u0634\u0631\u0648\u0639\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0627 \u062f\u0627\u0634\u0628\u0648\u0631\u062f Nitric \u062a\u0633\u062a \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0646\u0634\u0627\u0646\u06cc \u0648\u0628 \u062f\u0627\u0634\u0628\u0648\u0631\u062f \u0631\u0627 \u062f\u0631 \u062a\u0631\u0645\u06cc\u0646\u0627\u0644\u06cc \u06a9\u0647 \u0627\u0632 Nitric CLI \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc\u200c\u06a9\u0646\u062f\u060c \u0628\u06cc\u0627\u0628\u06cc\u062f. \u0628\u0647 \u0637\u0648\u0631 \u067e\u06cc\u0634 \u0641\u0631\u0636 http:\/\/localhost:49152 \u0627\u0633\u062a. \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0647 \u0628\u062f\u0646\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u062f \u0648 \u0622\u0646 \u0631\u0627 \u0628\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u0631\u0633\u0627\u0644 \u06a9\u0646\u06cc\u062f \/prompt \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc<\/p>\n<p>  \u062a\u0635\u0648\u06cc\u0631 \u0628\u0647 \u0627\u06cc\u0646\u062c\u0627 \u0645\u06cc \u0631\u0648\u062f<\/p>\n<p>  \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u062f\u0631 AWS<\/p>\n<p>\u0647\u0646\u06af\u0627\u0645\u06cc \u06a9\u0647 \u0622\u0645\u0627\u062f\u0647 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u067e\u0631\u0648\u0698\u0647 \u0647\u0633\u062a\u06cc\u062f\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u06cc\u06a9 \u0641\u0627\u06cc\u0644 \u067e\u0634\u062a\u0647 Nitric \u062c\u062f\u06cc\u062f \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u0645 \u06a9\u0647 AWS \u0631\u0627 \u0647\u062f\u0641 \u0642\u0631\u0627\u0631 \u0645\u06cc \u062f\u0647\u062f:<\/p>\n<p>nitric stack new dev aws<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>\u0641\u0627\u06cc\u0644 \u067e\u0634\u062a\u0647 \u0631\u0627 \u0628\u0647 \u0631\u0648\u0632 \u06a9\u0646\u06cc\u062f nitric.dev.yaml \u0628\u0627 \u0645\u0646\u0637\u0642\u0647 AWS \u0645\u0646\u0627\u0633\u0628 \u0648 \u062a\u062e\u0635\u06cc\u0635 \u062d\u0627\u0641\u0638\u0647 \u0628\u0631\u0627\u06cc \u0645\u062f\u06cc\u0631\u06cc\u062a \u0645\u062f\u0644.<\/p>\n<p>provider: nitric\/aws@1.14.0<br \/>\nregion: us-east-1<br \/>\nconfig:<br \/>\n  # How services will be deployed by default, if you have other services not running models<br \/>\n  # you can add them here too so they don&#8217;t use the same configuration<br \/>\n  default:<br \/>\n    lambda:<br \/>\n      # Set the memory to 6GB to handle the model, this automatically sets additional CPU allocation<br \/>\n      memory: 6144<br \/>\n      # Set a timeout of 30 seconds (this is the most API Gateway will wait for a response)<br \/>\n      timeout: 30<br \/>\n      # We add more storage to the lambda function, so it can store the model<br \/>\n      ephemeral-storage: 1024<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>\u0627\u0632 \u0622\u0646\u062c\u0627\u06cc\u06cc \u06a9\u0647 \u0645\u0627 \u0627\u0632 \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 \u067e\u06cc\u0634\u200c\u0641\u0631\u0636 Pulumi AWS Nitric \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc\u200c\u06a9\u0646\u06cc\u0645\u060c \u0645\u0637\u0645\u0626\u0646 \u0634\u0648\u06cc\u062f \u06a9\u0647 \u0628\u0631\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0622\u0646 \u062a\u0646\u0638\u06cc\u0645 \u0634\u062f\u0647\u200c\u0627\u06cc\u062f. \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0628\u06cc\u0634\u062a\u0631\u06cc \u062f\u0631 \u0645\u0648\u0631\u062f \u0646\u062d\u0648\u0647 \u0631\u0627\u0647\u200c\u0627\u0646\u062f\u0627\u0632\u06cc \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 AWS \u062f\u0631 \u0627\u0633\u0646\u0627\u062f \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 Nitric AWS \u0628\u06cc\u0627\u0628\u06cc\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0645\u06cc \u062e\u0648\u0627\u0647\u06cc\u062f \u0628\u0627 Terraform \u06cc\u0627 \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u0647 \u0627\u0628\u0631 \u062f\u06cc\u06af\u0631\u06cc \u0645\u0633\u062a\u0642\u0631 \u0634\u0648\u06cc\u062f\u060c \u0627\u06cc\u0646 \u0646\u06cc\u0632 \u0627\u0645\u06a9\u0627\u0646 \u067e\u0630\u06cc\u0631 \u0627\u0633\u062a. \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0628\u06cc\u0634\u062a\u0631\u06cc \u062f\u0631 \u0645\u0648\u0631\u062f \u0646\u062d\u0648\u0647 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 Nitric \u062f\u0631 \u0633\u0627\u06cc\u0631 \u067e\u0644\u062a\u0641\u0631\u0645 \u0647\u0627 \u062f\u0631 \u0627\u0633\u0646\u0627\u062f \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u06af\u0627\u0646 Nitric \u067e\u06cc\u062f\u0627 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0633\u067e\u0633 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u062f\u0633\u062a\u0648\u0631 \u0632\u06cc\u0631 \u0645\u0633\u062a\u0642\u0631 \u0634\u0648\u06cc\u062f:<\/p>\n<p>nitric up<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>\u0628\u0647 \u0646\u0634\u0627\u0646\u06cc \u0648\u0628 \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc API \u06a9\u0647 \u067e\u0633 \u0627\u0632 \u062a\u06a9\u0645\u06cc\u0644 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u062e\u0631\u0648\u062c\u06cc \u0645\u06cc\u200c\u0634\u0648\u062f\u060c \u062a\u0648\u062c\u0647 \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0628\u0639\u062f\u0627\u064b \u067e\u0631\u0648\u0698\u0647 \u0631\u0627 \u062a\u0645\u0627\u0645 \u06a9\u0631\u062f\u06cc\u062f\u060c \u0622\u0646 \u0631\u0627 \u062e\u0631\u0627\u0628 \u06a9\u0646\u06cc\u062f nitric down.<\/p>\n<p>  \u062a\u0633\u062a \u0631\u0648\u06cc AWS<\/p>\n<p>\u0628\u0631\u0627\u06cc \u0622\u0632\u0645\u0627\u06cc\u0634 \u0627\u06cc\u0646 \u0633\u0631\u0648\u06cc\u0633\u060c \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0632 \u0647\u0631 \u0627\u0628\u0632\u0627\u0631 \u062a\u0633\u062a API \u06a9\u0647 \u062f\u0648\u0633\u062a \u062f\u0627\u0631\u06cc\u062f\u060c \u0645\u0627\u0646\u0646\u062f cURL\u060c Postman \u0648 \u063a\u06cc\u0631\u0647 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f. \u062f\u0631 \u0627\u06cc\u0646\u062c\u0627 \u06cc\u06a9 \u0645\u062b\u0627\u0644 \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 cURL \u0622\u0648\u0631\u062f\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a:<\/p>\n<p>curl -X POST {your endpoint URL here}\/prompt -d &#8220;Hello, how are you?&#8221;<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>  \u067e\u0627\u0633\u062e \u0646\u0645\u0648\u0646\u0647<\/p>\n<p>\u067e\u0627\u0633\u062e \u0634\u0627\u0645\u0644 \u0646\u062a\u0627\u06cc\u062c\u060c \u0628\u0647 \u0639\u0644\u0627\u0648\u0647 \u0633\u0627\u06cc\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u0647\u0627 \u062e\u0648\u0627\u0647\u062f \u0628\u0648\u062f. \u062e\u0631\u0648\u062c\u06cc \u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646 \u062f\u0631 \u0622\u0631\u0627\u06cc\u0647 \u0627\u0646\u062a\u062e\u0627\u0628 \u06cc\u0627\u0641\u062a.<\/p>\n<p>{<br \/>\n  &#8220;id&#8221;: &#8220;cmpl-61064b38-45f9-496d-86d6-fdae4bc3db97&#8221;,<br \/>\n  &#8220;object&#8221;: &#8220;text_completion&#8221;,<br \/>\n  &#8220;created&#8221;: 1729655327,<br \/>\n  &#8220;model&#8221;: &#8220;.\/models\/Llama-3.2-1B-Instruct-Q4_K_M.gguf&#8221;,<br \/>\n  &#8220;choices&#8221;: [<br \/>\n    {<br \/>\n      &#8220;text&#8221;: &#8220;\\&#8221;I&#8217;m doing well, thank you for asking. I&#8217;m here and ready to assist you, so that&#8217;s a good start! How can I help you today?\\&#8221;&#8221;,<br \/>\n      &#8220;index&#8221;: 0,<br \/>\n      &#8220;logprobs&#8221;: null,<br \/>\n      &#8220;finish_reason&#8221;: &#8220;stop&#8221;<br \/>\n    }<br \/>\n  ],<br \/>\n  &#8220;usage&#8221;: {<br \/>\n    &#8220;prompt_tokens&#8221;: 26,<br \/>\n    &#8220;completion_tokens&#8221;: 33,<br \/>\n    &#8220;total_tokens&#8221;: 59<br \/>\n  }<br \/>\n}<\/p>\n<p>    \u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/p>\n<p>    \u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/p>\n<p>  \u062e\u0644\u0627\u0635\u0647<\/p>\n<p>\u0647\u0645\u0627\u0646\u0637\u0648\u0631 \u06a9\u0647 \u062f\u0631 \u0645\u062b\u0627\u0644 \u06a9\u062f \u0645\u0634\u0627\u0647\u062f\u0647 \u06a9\u0631\u062f\u06cc\u062f\u060c \u0645\u0627 \u06cc\u06a9 \u0633\u0627\u062e\u062a\u0627\u0631 \u0627\u0639\u0644\u0627\u0646 \u0646\u0633\u0628\u062a\u0627\u064b \u0627\u0628\u062a\u062f\u0627\u06cc\u06cc \u062a\u0646\u0638\u06cc\u0645 \u06a9\u0631\u062f\u0647\u200c\u0627\u06cc\u0645\u060c \u0627\u0645\u0627 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u06cc\u0646 \u0633\u0627\u062e\u062a\u0627\u0631 \u0631\u0627 \u06af\u0633\u062a\u0631\u0634 \u062f\u0647\u06cc\u062f \u062a\u0627 \u0627\u0639\u0644\u0627\u0646\u200c\u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631\u06cc \u0631\u0627 \u0634\u0627\u0645\u0644 \u0634\u0648\u062f\u060c \u0627\u0632 \u062c\u0645\u0644\u0647 \u0627\u0639\u0644\u0627\u0646\u200c\u0647\u0627\u06cc \u0633\u06cc\u0633\u062a\u0645\u06cc \u06a9\u0647 \u0628\u0647 \u0645\u062d\u062f\u0648\u062f \u06a9\u0631\u062f\u0646\/\u0647\u062f\u0627\u06cc\u062a \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0645\u062f\u0644 \u06cc\u0627 \u062d\u062a\u06cc \u062a\u0639\u0627\u0645\u0644\u0627\u062a \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631 \u0628\u0627 \u0645\u062f\u0644 \u0647\u0645\u0686\u0646\u06cc\u0646\u060c \u062f\u0631 \u0627\u06cc\u0646 \u0645\u062b\u0627\u0644\u060c \u0645\u0627 \u0645\u062f\u0644 \u0631\u0627 \u0645\u0633\u062a\u0642\u06cc\u0645\u0627\u064b \u0628\u0647\u200c\u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 API \u062f\u0631 \u0645\u0639\u0631\u0636 \u062f\u06cc\u062f \u0642\u0631\u0627\u0631 \u0645\u06cc\u200c\u062f\u0647\u06cc\u0645\u060c \u0627\u0645\u0627 \u0627\u06cc\u0646 \u0632\u0645\u0627\u0646 \u067e\u0627\u0633\u062e\u06af\u0648\u06cc\u06cc \u0631\u0627 \u0628\u0647 30 \u062b\u0627\u0646\u06cc\u0647 \u062f\u0631 AWS \u0628\u0627 API Gateway \u0645\u062d\u062f\u0648\u062f \u0645\u06cc\u200c\u06a9\u0646\u062f.<\/p>\n<p>\u062f\u0631 \u0631\u0627\u0647\u0646\u0645\u0627\u0647\u0627\u06cc \u0622\u06cc\u0646\u062f\u0647\u060c \u0646\u0634\u0627\u0646 \u062e\u0648\u0627\u0647\u06cc\u0645 \u062f\u0627\u062f \u06a9\u0647 \u0686\u06af\u0648\u0646\u0647 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0641\u0631\u0627\u062a\u0631 \u0627\u0632 \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0633\u0627\u062f\u0647 \u06cc\u06a9\u200c\u0628\u0627\u0631\u0647 \u0628\u0647 \u062a\u0639\u0627\u0645\u0644\u200c\u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631\u060c \u0645\u0627\u0646\u0646\u062f \u062d\u0641\u0638 \u0628\u0627\u0641\u062a \u0628\u06cc\u0646 \u062f\u0631\u062e\u0648\u0627\u0633\u062a\u200c\u0647\u0627 \u0628\u0631\u0648\u06cc\u062f. \u0645\u0627 \u0647\u0645\u0686\u0646\u06cc\u0646 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u0645 Websockets \u0648 \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u062c\u0631\u06cc\u0627\u0646\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc \u0627\u0631\u0627\u0626\u0647 \u062a\u062c\u0631\u0628\u0647 \u06a9\u0627\u0631\u0628\u0631\u06cc \u0628\u0647\u062a\u0631 \u0628\u0631\u0627\u06cc \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0628\u0632\u0631\u06af\u200c\u062a\u0631 \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u0645.<\/p>\n<div data-article-id=\"2120912\" id=\"article-body\">\n<p>Llama 3.2 1B \u06cc\u06a9 \u0645\u062f\u0644 \u0647\u0648\u0634 \u0645\u0635\u0646\u0648\u0639\u06cc \u0633\u0628\u06a9 \u0648\u0632\u0646 \u0627\u0633\u062a \u06a9\u0647 \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u062c\u0627\u0644\u0628 \u0645\u06cc \u06a9\u0646\u062f \u0632\u06cc\u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646 \u0622\u0646 \u0631\u0627 \u0646\u0633\u0628\u062a\u0627\u064b \u0633\u0631\u06cc\u0639 \u0628\u062f\u0648\u0646 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0634\u062a\u0627\u0628 GPU \u0627\u062c\u0631\u0627 \u06a9\u0631\u062f.<\/p>\n<p>\u0645\u0627 \u0627\u0632 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc\u06cc \u0627\u0632 Hugging Face \u0648 Nitric \u0628\u0631\u0627\u06cc \u0646\u0634\u0627\u0646 \u062f\u0627\u062f\u0646 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0622\u0646 \u0648 \u0645\u062f\u06cc\u0631\u06cc\u062a \u0632\u06cc\u0631\u0633\u0627\u062e\u062a\u200c\u0647\u0627\u06cc \u0627\u0637\u0631\u0627\u0641\u060c \u0645\u0627\u0646\u0646\u062f \u0645\u0633\u06cc\u0631\u0647\u0627\u06cc API \u0648 \u0627\u0633\u062a\u0642\u0631\u0627\u0631\u060c \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n<p>\u067e\u06cc\u0634 \u0646\u06cc\u0627\u0632\u0647\u0627<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_85 counter-hierarchy ez-toc-counter-rtl ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">\u0641\u0647\u0631\u0633\u062a \u0645\u0637\u0627\u0644\u0628<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"\u062a\u063a\u06cc\u06cc\u0631 \u0648\u0636\u0639\u06cc\u062a \u0641\u0647\u0631\u0633\u062a \u0645\u0637\u0627\u0644\u0628\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%B1%D8%A7%D9%87_%D8%A7%D9%86%D8%AF%D8%A7%D8%B2%DB%8C_%D9%BE%D8%B1%D9%88%DA%98%D9%87\" >\u0631\u0627\u0647 \u0627\u0646\u062f\u0627\u0632\u06cc \u067e\u0631\u0648\u0698\u0647<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D9%85%D8%AF%D9%84_%D9%84%D8%A7%D9%85%D8%A7_%D8%B1%D8%A7_%D8%A7%D9%86%D8%AA%D8%AE%D8%A7%D8%A8_%DA%A9%D9%86%DB%8C%D8%AF\" >\u0645\u062f\u0644 \u0644\u0627\u0645\u0627 \u0631\u0627 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%DB%8C%DA%A9_%D8%B3%D8%B1%D9%88%DB%8C%D8%B3_%D8%A8%D8%B1%D8%A7%DB%8C_%D8%A7%D8%AC%D8%B1%D8%A7%DB%8C_%D9%85%D8%AF%D9%84_%D8%A7%DB%8C%D8%AC%D8%A7%D8%AF_%DA%A9%D9%86%DB%8C%D8%AF\" >\u06cc\u06a9 \u0633\u0631\u0648\u06cc\u0633 \u0628\u0631\u0627\u06cc \u0627\u062c\u0631\u0627\u06cc \u0645\u062f\u0644 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%AE%D9%88%D8%A8%D8%8C_%D8%A8%DB%8C%D8%A7%DB%8C%DB%8C%D8%AF_%D8%A7%DB%8C%D9%86_%DA%A9%D8%A7%D8%B1_%D8%B1%D8%A7_%D8%A7%D8%AC%D8%B1%D8%A7_%DA%A9%D9%86%DB%8C%D9%85\" >\u062e\u0648\u0628\u060c \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0627\u06cc\u0646 \u06a9\u0627\u0631 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u0645!<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%AA%D8%B5%D9%88%DB%8C%D8%B1_%D8%A8%D9%87_%D8%A7%DB%8C%D9%86%D8%AC%D8%A7_%D9%85%DB%8C_%D8%B1%D9%88%D8%AF\" >\u062a\u0635\u0648\u06cc\u0631 \u0628\u0647 \u0627\u06cc\u0646\u062c\u0627 \u0645\u06cc \u0631\u0648\u062f<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%A7%D8%B3%D8%AA%D9%82%D8%B1%D8%A7%D8%B1_%D8%AF%D8%B1_AWS\" >\u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u062f\u0631 AWS<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%AA%D8%B3%D8%AA_%D8%B1%D9%88%DB%8C_AWS\" >\u062a\u0633\u062a \u0631\u0648\u06cc AWS<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D9%BE%D8%A7%D8%B3%D8%AE_%D9%86%D9%85%D9%88%D9%86%D9%87\" >\u067e\u0627\u0633\u062e \u0646\u0645\u0648\u0646\u0647<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/nabfollower.com\/blog\/llama-32-on-lambda-k9p\/#%D8%AE%D9%84%D8%A7%D8%B5%D9%87\" >\u062e\u0644\u0627\u0635\u0647<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"%D8%B1%D8%A7%D9%87_%D8%A7%D9%86%D8%AF%D8%A7%D8%B2%DB%8C_%D9%BE%D8%B1%D9%88%DA%98%D9%87\"><\/span>\n<p>  \u0631\u0627\u0647 \u0627\u0646\u062f\u0627\u0632\u06cc \u067e\u0631\u0648\u0698\u0647<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0628\u0627 \u0627\u06cc\u062c\u0627\u062f \u06cc\u06a9 \u067e\u0631\u0648\u0698\u0647 \u062c\u062f\u06cc\u062f \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0642\u0627\u0644\u0628 \u0634\u0631\u0648\u0639 \u067e\u0627\u06cc\u062a\u0648\u0646 Nitric \u0634\u0631\u0648\u0639 \u06a9\u0646\u06cc\u0645.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>nitric new llama py-starter\n<span class=\"nb\">cd <\/span>llama\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u062f\u0631 \u0645\u0631\u062d\u0644\u0647 \u0628\u0639\u062f\u060c \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u067e\u0627\u06cc\u0647 \u0631\u0627 \u0646\u0635\u0628 \u06a9\u0646\u06cc\u0645\u060c \u0633\u067e\u0633 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u0627\u0636\u0627\u0641\u06cc \u0631\u0627 \u06a9\u0647 \u0628\u0647 \u0637\u0648\u0631 \u062e\u0627\u0635 \u0628\u0631\u0627\u06cc \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u0645\u062f\u0644 \u0632\u0628\u0627\u0646 \u0646\u06cc\u0627\u0632 \u062f\u0627\u0631\u06cc\u0645\u060c \u0627\u0636\u0627\u0641\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u0645.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\"># Install the base dependencies<\/span>\nuv <span class=\"nb\">sync<\/span>\n<span class=\"c\"># Add the llama-cpp-python dependency<\/span>\nuv add llama-cpp-python\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"%D9%85%D8%AF%D9%84_%D9%84%D8%A7%D9%85%D8%A7_%D8%B1%D8%A7_%D8%A7%D9%86%D8%AA%D8%AE%D8%A7%D8%A8_%DA%A9%D9%86%DB%8C%D8%AF\"><\/span>\n<p>  \u0645\u062f\u0644 \u0644\u0627\u0645\u0627 \u0631\u0627 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Llama 3.2 \u062f\u0631 \u0627\u0646\u062f\u0627\u0632\u0647\u200c\u0647\u0627 \u0648 \u067e\u06cc\u06a9\u0631\u0628\u0646\u062f\u06cc\u200c\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0645\u0648\u062c\u0648\u062f \u0627\u0633\u062a \u06a9\u0647 \u0647\u0631 \u06a9\u062f\u0627\u0645 \u0627\u0632 \u0646\u0638\u0631 \u0639\u0645\u0644\u06a9\u0631\u062f\u060c \u062f\u0642\u062a \u0648 \u0645\u0646\u0627\u0628\u0639 \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632\u060c \u0645\u0639\u0627\u0648\u0636\u0647\u200c\u0647\u0627\u06cc \u062e\u0627\u0635 \u062e\u0648\u062f \u0631\u0627 \u062f\u0627\u0631\u0646\u062f. \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u0628\u062f\u0648\u0646 \u0634\u062a\u0627\u0628 GPU\u060c \u0645\u0627\u0646\u0646\u062f AWS Lambda\u060c \u0645\u0647\u0645 \u0627\u0633\u062a \u06a9\u0647 \u0645\u062f\u0644\u06cc \u0631\u0627 \u0627\u0646\u062a\u062e\u0627\u0628 \u06a9\u0646\u06cc\u062f \u06a9\u0647 \u0633\u0628\u06a9 \u0648 \u06a9\u0627\u0631\u0622\u0645\u062f \u0628\u0627\u0634\u062f \u062a\u0627 \u0627\u0637\u0645\u06cc\u0646\u0627\u0646 \u062d\u0627\u0635\u0644 \u0634\u0648\u062f \u06a9\u0647 \u062f\u0631 \u0645\u062d\u062f\u0648\u062f\u06cc\u062a \u0647\u0627\u06cc \u0622\u0646 \u0645\u062d\u06cc\u0637 \u0627\u062c\u0631\u0627 \u0645\u06cc \u0634\u0648\u062f.<\/p>\n<p>\u0645\u0627 \u0627\u0632 \u06cc\u06a9 \u0646\u0633\u062e\u0647 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0647 \u0634\u062f\u0647 \u0627\u0632 \u0645\u062f\u0644 \u0633\u0628\u06a9 \u0648\u0632\u0646 Llama 1B\u060c \u0628\u0647 \u0648\u06cc\u0698\u0647 Llama-3.2-1B-Instruct-Q4_K_M.gguf \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u062e\u0648\u0627\u0647\u06cc\u0645 \u06a9\u0631\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0628\u0627 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0627\u0633\u06cc\u0648\u0646 \u0622\u0634\u0646\u0627\u06cc\u06cc \u0646\u062f\u0627\u0631\u06cc\u062f\u060c \u0627\u06cc\u0646 \u062a\u06a9\u0646\u06cc\u06a9\u06cc \u0627\u0633\u062a \u06a9\u0647 \u0627\u0646\u062f\u0627\u0632\u0647 \u0648 \u0645\u0646\u0627\u0628\u0639 \u0645\u0648\u0631\u062f \u0646\u06cc\u0627\u0632 \u0645\u062f\u0644 \u0631\u0627 \u06a9\u0627\u0647\u0634 \u0645\u06cc\u200c\u062f\u0647\u062f\u060c \u06a9\u0647 \u062f\u0631 \u0645\u0648\u0631\u062f \u0645\u0627\u060c \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647\u200c\u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u0645\u0646\u0627\u0633\u0628 \u0645\u06cc\u200c\u06a9\u0646\u062f \u0627\u0645\u0627 \u0645\u0645\u06a9\u0646 \u0627\u0633\u062a \u0628\u0631 \u062f\u0642\u062a \u0645\u062f\u0644 \u062a\u0623\u062b\u06cc\u0631 \u0628\u06af\u0630\u0627\u0631\u062f.<\/p>\n<p>\u062a\u06cc\u0645 LM Studio \u0686\u0646\u062f\u06cc\u0646 \u0646\u0633\u062e\u0647 \u06a9\u0648\u0627\u0646\u062a\u06cc\u0632\u0647 \u0634\u062f\u0647 Llama 3.2 1B \u0631\u0627 \u062f\u0631 Hugging Face \u0627\u0631\u0627\u0626\u0647 \u0645\u06cc \u06a9\u0646\u062f. \u0633\u0639\u06cc \u06a9\u0646\u06cc\u062f \u0646\u0633\u062e\u0647\u200c\u0647\u0627\u06cc \u0645\u062e\u062a\u0644\u0641 \u0631\u0627 \u0627\u0645\u062a\u062d\u0627\u0646 \u06a9\u0646\u06cc\u062f \u062a\u0627 \u0628\u0647\u062a\u0631\u06cc\u0646 \u0645\u0637\u0627\u0628\u0642 \u0628\u0627 \u0646\u06cc\u0627\u0632\u0647\u0627\u06cc \u0634\u0645\u0627 \u0631\u0627 \u0628\u06cc\u0627\u0628\u06cc\u062f\u060c \u0645\u0627\u0646\u0646\u062f Q5_K_M\u060c \u06a9\u0647 \u06a9\u0645\u06cc \u0628\u0632\u0631\u06af\u062a\u0631 \u0627\u0633\u062a \u0627\u0645\u0627 \u06a9\u06cc\u0641\u06cc\u062a \u0628\u0627\u0644\u0627\u062a\u0631\u06cc \u062f\u0627\u0631\u062f.<\/p>\n<p>\u0628\u06cc\u0627\u06cc\u06cc\u062f \u0645\u062f\u0644 \u0627\u0646\u062a\u062e\u0627\u0628 \u0634\u062f\u0647 \u0631\u0627 \u062f\u0627\u0646\u0644\u0648\u062f \u06a9\u0631\u062f\u0647 \u0648 \u062f\u0631 \u067e\u0648\u0634\u0647 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc \u067e\u0631\u0648\u0698\u0647 \u062e\u0648\u062f \u0630\u062e\u06cc\u0631\u0647 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0644\u06cc\u0646\u06a9 \u062f\u0627\u0646\u0644\u0648\u062f Llama-3.2-1B-Instruct-Q4_K_M.gguf:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nb\">mkdir <\/span>models\n<span class=\"nb\">cd <\/span>models\n<span class=\"c\"># This model is 0.81GB, it may take a little while to download<\/span>\ncurl <span class=\"nt\">-OL<\/span> https:\/\/huggingface.co\/bartowski\/Llama-3.2-1B-Instruct-GGUF\/resolve\/main\/Llama-3.2-1B-Instruct-Q4_K_M.gguf\n<span class=\"nb\">cd<\/span> ..\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0633\u0627\u062e\u062a\u0627\u0631 \u067e\u0648\u0634\u0647 \u0634\u0645\u0627 \u0628\u0627\u06cc\u062f \u0628\u0647 \u0634\u06a9\u0644 \u0632\u06cc\u0631 \u0628\u0627\u0634\u062f:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>\/llama\n    \/models\n        Llama-3.2-1B-Instruct-Q4_K_M.gguf\n    \/services\n        api.py\n    nitric.yaml\n    pyproject.toml\n    python.dockerfile\n    python.dockerfile.ignore\n    README.md\n    uv.lock\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"%DB%8C%DA%A9_%D8%B3%D8%B1%D9%88%DB%8C%D8%B3_%D8%A8%D8%B1%D8%A7%DB%8C_%D8%A7%D8%AC%D8%B1%D8%A7%DB%8C_%D9%85%D8%AF%D9%84_%D8%A7%DB%8C%D8%AC%D8%A7%D8%AF_%DA%A9%D9%86%DB%8C%D8%AF\"><\/span>\n<p>  \u06cc\u06a9 \u0633\u0631\u0648\u06cc\u0633 \u0628\u0631\u0627\u06cc \u0627\u062c\u0631\u0627\u06cc \u0645\u062f\u0644 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u062f<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u062f\u0631 \u0645\u0631\u062d\u0644\u0647 \u0628\u0639\u062f\u060c \u0627\u0632 Nitric \u0628\u0631\u0627\u06cc \u0627\u06cc\u062c\u0627\u062f \u06cc\u06a9 API HTTP \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u06cc\u0645 \u06a9\u0647 \u0628\u0647 \u0634\u0645\u0627 \u0627\u0645\u06a9\u0627\u0646 \u0645\u06cc \u062f\u0647\u062f \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0647\u0627\u06cc\u06cc \u0631\u0627 \u0628\u0647 \u0645\u062f\u0644 Llama \u0627\u0631\u0633\u0627\u0644 \u06a9\u0646\u06cc\u062f \u0648 \u062e\u0631\u0648\u062c\u06cc \u0631\u0627 \u062f\u0631 \u06cc\u06a9 \u067e\u0627\u0633\u062e \u062f\u0631\u06cc\u0627\u0641\u062a \u06a9\u0646\u06cc\u062f. API \u062e\u0631\u0648\u062c\u06cc \u062e\u0627\u0645 \u0631\u0627 \u0627\u0632 \u0645\u062f\u0644 \u0628\u0631\u0645\u06cc \u06af\u0631\u062f\u0627\u0646\u062f\u060c \u0627\u0645\u0627 \u0634\u0645\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0647 \u062f\u0644\u062e\u0648\u0627\u0647 \u062a\u0646\u0638\u06cc\u0645 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0645\u062d\u062a\u0648\u06cc\u0627\u062a \u0631\u0627 \u062c\u0627\u06cc\u06af\u0632\u06cc\u0646 \u06a9\u0646\u06cc\u062f <code>services\/api.py<\/code> \u0628\u0627 \u06a9\u062f \u0632\u06cc\u0631 \u06a9\u0647 \u0645\u062f\u0644 Llama \u0631\u0627 \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u06a9\u0631\u062f\u0647 \u0648 \u0639\u0645\u0644\u06a9\u0631\u062f \u0633\u0631\u06cc\u0639 \u0631\u0627 \u067e\u06cc\u0627\u062f\u0647 \u0633\u0627\u0632\u06cc \u0645\u06cc \u06a9\u0646\u062f. \u0628\u0631\u0627\u06cc \u062f\u0631\u06a9 \u06a9\u062f \u06a9\u0645\u06cc \u0632\u0645\u0627\u0646 \u0628\u06af\u0630\u0627\u0631\u06cc\u062f. \u06cc\u06a9 API \u0631\u0627 \u0628\u0627 \u06cc\u06a9 \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc \/ prompt \u062a\u0639\u0631\u06cc\u0641 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a POST \u0631\u0627 \u0628\u0627 \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u062f\u0631 \u0628\u062f\u0646\u0647 \u0645\u06cc \u067e\u0630\u06cc\u0631\u062f. \u0627\u06cc\u0646 <code>process_prompt<\/code> \u062a\u0627\u0628\u0639 \u062f\u0633\u062a\u0648\u0631 \u0631\u0627 \u0628\u0647 \u0645\u062f\u0644 Llama \u0645\u06cc \u0641\u0631\u0633\u062a\u062f \u0648 \u067e\u0627\u0633\u062e \u0631\u0627 \u0628\u0631\u0645\u06cc \u06af\u0631\u062f\u0627\u0646\u062f.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"kn\">from<\/span> <span class=\"n\">nitric.resources<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">api<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">nitric.application<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Nitric<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">nitric.context<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">HttpContext<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">llama_cpp<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Llama<\/span>\n\n<span class=\"c1\"># Load the locally stored Llama model\n<\/span><span class=\"n\">llm<\/span> <span class=\"o\">=<\/span> <span class=\"nc\">Llama<\/span><span class=\"p\">(<\/span><span class=\"n\">model_path<\/span><span class=\"o\">=<\/span><span class=\"sh\">\"<\/span><span class=\"s\">.\/models\/Llama-3.2-1B-Instruct-Q4_K_M.gguf<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n<span class=\"c1\"># Function to execute a prompt using the Llama model\n<\/span><span class=\"k\">def<\/span> <span class=\"nf\">process_prompt<\/span><span class=\"p\">(<\/span><span class=\"n\">user_prompt<\/span><span class=\"p\">):<\/span>\n    <span class=\"n\">system_prompt<\/span> <span class=\"o\">=<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">You are a helpful assistant.<\/span><span class=\"sh\">\"<\/span>\n\n    <span class=\"c1\"># See https:\/\/www.llama.com\/docs\/model-cards-and-prompt-formats\/llama3_1\/ for details about prompt format\n<\/span>    <span class=\"n\">prompt<\/span> <span class=\"o\">=<\/span> <span class=\"p\">(<\/span>\n        <span class=\"c1\"># System Prompt\n<\/span>        <span class=\"sa\">f<\/span><span class=\"sh\">'<\/span><span class=\"s\">&lt;|start_header_id|&gt;system&lt;|end_header_id|&gt;<\/span><span class=\"si\">{<\/span><span class=\"n\">system_prompt<\/span><span class=\"si\">}<\/span><span class=\"s\">&lt;|eot_id|&gt;<\/span><span class=\"sh\">'<\/span>\n        <span class=\"c1\"># User Prompt\n<\/span>        <span class=\"sa\">f<\/span><span class=\"sh\">'<\/span><span class=\"s\">&lt;|start_header_id|&gt;user&lt;|end_header_id|&gt;<\/span><span class=\"si\">{<\/span><span class=\"n\">user_prompt<\/span><span class=\"si\">}<\/span><span class=\"s\">&lt;|eot_id|&gt;<\/span><span class=\"sh\">'<\/span>\n        <span class=\"c1\"># Start assistants turn (we leave this open ended as the assistant hasn't started its turn)\n<\/span>        <span class=\"sa\">f<\/span><span class=\"sh\">'<\/span><span class=\"s\">&lt;|start_header_id|&gt;assistant&lt;|end_header_id|&gt;<\/span><span class=\"sh\">'<\/span>\n    <span class=\"p\">)<\/span>\n\n    <span class=\"n\">response<\/span> <span class=\"o\">=<\/span> <span class=\"nf\">llm<\/span><span class=\"p\">(<\/span>\n        <span class=\"n\">prompt<\/span><span class=\"o\">=<\/span><span class=\"n\">prompt<\/span><span class=\"p\">,<\/span>\n        <span class=\"c1\"># Unlimited, consider setting a token limit\n<\/span>        <span class=\"n\">max_tokens<\/span><span class=\"o\">=-<\/span><span class=\"mi\">1<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">temperature<\/span><span class=\"o\">=<\/span><span class=\"mf\">0.7<\/span><span class=\"p\">,<\/span>\n    <span class=\"p\">)<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"n\">response<\/span>\n\n<span class=\"c1\"># Define an API for the prompt service\n<\/span><span class=\"n\">main<\/span> <span class=\"o\">=<\/span> <span class=\"nf\">api<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">main<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n\n<span class=\"nd\">@main.post<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">\/prompt<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">async<\/span> <span class=\"k\">def<\/span> <span class=\"nf\">handle_prompt<\/span><span class=\"p\">(<\/span><span class=\"n\">ctx<\/span><span class=\"p\">:<\/span> <span class=\"n\">HttpContext<\/span><span class=\"p\">):<\/span>\n    <span class=\"c1\"># assume the input is text\/plain\n<\/span>    <span class=\"n\">prompt<\/span> <span class=\"o\">=<\/span> <span class=\"n\">ctx<\/span><span class=\"p\">.<\/span><span class=\"n\">req<\/span><span class=\"p\">.<\/span><span class=\"n\">data<\/span>\n\n    <span class=\"k\">try<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">ctx<\/span><span class=\"p\">.<\/span><span class=\"n\">res<\/span><span class=\"p\">.<\/span><span class=\"n\">body<\/span> <span class=\"o\">=<\/span> <span class=\"nf\">process_prompt<\/span><span class=\"p\">(<\/span><span class=\"n\">prompt<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">except<\/span> <span class=\"nb\">Exception<\/span> <span class=\"k\">as<\/span> <span class=\"n\">e<\/span><span class=\"p\">:<\/span>\n        <span class=\"nf\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Error processing prompt: <\/span><span class=\"si\">{<\/span><span class=\"n\">e<\/span><span class=\"si\">}<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">ctx<\/span><span class=\"p\">.<\/span><span class=\"n\">res<\/span><span class=\"p\">.<\/span><span class=\"n\">body<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span><span class=\"sh\">\"<\/span><span class=\"s\">error<\/span><span class=\"sh\">\"<\/span><span class=\"p\">:<\/span> <span class=\"nf\">str<\/span><span class=\"p\">(<\/span><span class=\"n\">e<\/span><span class=\"p\">)}<\/span>\n        <span class=\"n\">ctx<\/span><span class=\"p\">.<\/span><span class=\"n\">res<\/span><span class=\"p\">.<\/span><span class=\"n\">status<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">500<\/span>\n\n<span class=\"n\">Nitric<\/span><span class=\"p\">.<\/span><span class=\"nf\">run<\/span><span class=\"p\">()<\/span>\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"%D8%AE%D9%88%D8%A8%D8%8C_%D8%A8%DB%8C%D8%A7%DB%8C%DB%8C%D8%AF_%D8%A7%DB%8C%D9%86_%DA%A9%D8%A7%D8%B1_%D8%B1%D8%A7_%D8%A7%D8%AC%D8%B1%D8%A7_%DA%A9%D9%86%DB%8C%D9%85\"><\/span>\n<p>  \u062e\u0648\u0628\u060c \u0628\u06cc\u0627\u06cc\u06cc\u062f \u0627\u06cc\u0646 \u06a9\u0627\u0631 \u0631\u0627 \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u0645!<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0627\u06a9\u0646\u0648\u0646 \u06a9\u0647 \u06cc\u06a9 API \u062a\u0639\u0631\u06cc\u0641 \u06a9\u0631\u062f\u0647 \u0627\u06cc\u062f\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u0622\u0646 \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u0645\u062d\u0644\u06cc \u0622\u0632\u0645\u0627\u06cc\u0634 \u06a9\u0646\u06cc\u0645. \u0627\u0644\u06af\u0648\u06cc \u0634\u0631\u0648\u0639 \u067e\u0627\u06cc\u062a\u0648\u0646 \u0627\u0632 python3.11-bookworm-slim \u0628\u0647 \u0639\u0646\u0648\u0627\u0646 \u062a\u0635\u0648\u06cc\u0631 \u0645\u062d\u0641\u0638\u0647 \u0627\u0635\u0644\u06cc \u062e\u0648\u062f \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0645\u0646\u0627\u0633\u0628\u06cc \u0628\u0631\u0627\u06cc \u0628\u0627\u0631\u06af\u0630\u0627\u0631\u06cc \u0645\u062f\u0644 Llama \u0646\u062f\u0627\u0631\u062f. \u0627\u062c\u0627\u0632\u0647 \u062f\u0647\u06cc\u062f Dockerfile \u0631\u0627 \u0628\u0647 \u0631\u0648\u0632 \u06a9\u0646\u06cc\u0645 \u062a\u0627 \u0628\u0647 \u062c\u0627\u06cc \u0622\u0646 \u0627\u0632 Python3.11-bookworm (\u0646\u0633\u062e\u0647 \u063a\u06cc\u0631 \u0628\u0627\u0631\u06cc\u06a9) \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u0645.<\/p>\n<p>\u0628\u0647 \u0631\u0648\u0632 \u0631\u0633\u0627\u0646\u06cc \u062e\u0637 2:<\/p>\n<p>FROM python:3.11-bookworm<br \/>1<br \/>FROM python:3.11-bookworm<\/p>\n<p>\u0628\u0647 \u0631\u0648\u0632 \u0631\u0633\u0627\u0646\u06cc \u062e\u0637 19:<\/p>\n<p>FROM python:3.11-bookworm<br \/>1<br \/>FROM python:3.11-bookworm<\/p>\n<p>\u0627\u06a9\u0646\u0648\u0646 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u062e\u062f\u0645\u0627\u062a \u062e\u0648\u062f \u0631\u0627 \u0628\u0647 \u0635\u0648\u0631\u062a \u0645\u062d\u0644\u06cc \u0627\u062c\u0631\u0627 \u06a9\u0646\u06cc\u0645:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>nitric run\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p><code>nitric run<\/code>  \u0628\u0631\u0646\u0627\u0645\u0647 \u0634\u0645\u0627 \u0631\u0627 \u062f\u0631 \u0638\u0631\u0641\u06cc \u0634\u0631\u0648\u0639 \u0645\u06cc \u06a9\u0646\u062f \u06a9\u0647 \u0634\u0627\u0645\u0644 \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 llama_cpp \u0627\u0633\u062a. \u0627\u06af\u0631 \u062a\u0631\u062c\u06cc\u062d \u0645\u06cc \u062f\u0647\u06cc\u062f \u0627\u0632 \u0646\u06cc\u062a\u0631\u06cc\u06a9 \u0627\u0633\u062a\u0627\u0631\u062a \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f\u060c \u0628\u0627\u06cc\u062f \u0648\u0627\u0628\u0633\u062a\u06af\u06cc \u0647\u0627\u06cc\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc llama-cpp-python \u0645\u0627\u0646\u0646\u062f CMake \u0648 LLVM \u0646\u0635\u0628 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u067e\u0633 \u0627\u0632 \u0634\u0631\u0648\u0639\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0622\u0646 \u0631\u0627 \u0628\u0627 \u062f\u0627\u0634\u0628\u0648\u0631\u062f Nitric \u062a\u0633\u062a \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0646\u0634\u0627\u0646\u06cc \u0648\u0628 \u062f\u0627\u0634\u0628\u0648\u0631\u062f \u0631\u0627 \u062f\u0631 \u062a\u0631\u0645\u06cc\u0646\u0627\u0644\u06cc \u06a9\u0647 \u0627\u0632 Nitric CLI \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc\u200c\u06a9\u0646\u062f\u060c \u0628\u06cc\u0627\u0628\u06cc\u062f. \u0628\u0647 \u0637\u0648\u0631 \u067e\u06cc\u0634 \u0641\u0631\u0636 http:\/\/localhost:49152 \u0627\u0633\u062a. \u06cc\u06a9 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0628\u0647 \u0628\u062f\u0646\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u062f \u0648 \u0622\u0646 \u0631\u0627 \u0628\u0647 \u062f\u0631\u062e\u0648\u0627\u0633\u062a \u0627\u0631\u0633\u0627\u0644 \u06a9\u0646\u06cc\u062f <code>\/prompt<\/code> \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc<\/p>\n<h2><span class=\"ez-toc-section\" id=\"%D8%AA%D8%B5%D9%88%DB%8C%D8%B1_%D8%A8%D9%87_%D8%A7%DB%8C%D9%86%D8%AC%D8%A7_%D9%85%DB%8C_%D8%B1%D9%88%D8%AF\"><\/span>\n<p>  \u062a\u0635\u0648\u06cc\u0631 \u0628\u0647 \u0627\u06cc\u0646\u062c\u0627 \u0645\u06cc \u0631\u0648\u062f<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h2><span class=\"ez-toc-section\" id=\"%D8%A7%D8%B3%D8%AA%D9%82%D8%B1%D8%A7%D8%B1_%D8%AF%D8%B1_AWS\"><\/span>\n<p>  \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u062f\u0631 AWS<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0647\u0646\u06af\u0627\u0645\u06cc \u06a9\u0647 \u0622\u0645\u0627\u062f\u0647 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u067e\u0631\u0648\u0698\u0647 \u0647\u0633\u062a\u06cc\u062f\u060c \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u0645 \u06cc\u06a9 \u0641\u0627\u06cc\u0644 \u067e\u0634\u062a\u0647 Nitric \u062c\u062f\u06cc\u062f \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u06cc\u0645 \u06a9\u0647 AWS \u0631\u0627 \u0647\u062f\u0641 \u0642\u0631\u0627\u0631 \u0645\u06cc \u062f\u0647\u062f:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>nitric stack new dev aws\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0641\u0627\u06cc\u0644 \u067e\u0634\u062a\u0647 \u0631\u0627 \u0628\u0647 \u0631\u0648\u0632 \u06a9\u0646\u06cc\u062f <code>nitric.dev.yaml<\/code> \u0628\u0627 \u0645\u0646\u0637\u0642\u0647 AWS \u0645\u0646\u0627\u0633\u0628 \u0648 \u062a\u062e\u0635\u06cc\u0635 \u062d\u0627\u0641\u0638\u0647 \u0628\u0631\u0627\u06cc \u0645\u062f\u06cc\u0631\u06cc\u062a \u0645\u062f\u0644.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">provider<\/span><span class=\"pi\">:<\/span> <span class=\"s\">nitric\/aws@1.14.0<\/span>\n<span class=\"na\">region<\/span><span class=\"pi\">:<\/span> <span class=\"s\">us-east-1<\/span>\n<span class=\"na\">config<\/span><span class=\"pi\">:<\/span>\n  <span class=\"c1\"># How services will be deployed by default, if you have other services not running models<\/span>\n  <span class=\"c1\"># you can add them here too so they don't use the same configuration<\/span>\n  <span class=\"na\">default<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">lambda<\/span><span class=\"pi\">:<\/span>\n      <span class=\"c1\"># Set the memory to 6GB to handle the model, this automatically sets additional CPU allocation<\/span>\n      <span class=\"na\">memory<\/span><span class=\"pi\">:<\/span> <span class=\"m\">6144<\/span>\n      <span class=\"c1\"># Set a timeout of 30 seconds (this is the most API Gateway will wait for a response)<\/span>\n      <span class=\"na\">timeout<\/span><span class=\"pi\">:<\/span> <span class=\"m\">30<\/span>\n      <span class=\"c1\"># We add more storage to the lambda function, so it can store the model<\/span>\n      <span class=\"na\">ephemeral-storage<\/span><span class=\"pi\">:<\/span> <span class=\"m\">1024<\/span>\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0627\u0632 \u0622\u0646\u062c\u0627\u06cc\u06cc \u06a9\u0647 \u0645\u0627 \u0627\u0632 \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 \u067e\u06cc\u0634\u200c\u0641\u0631\u0636 Pulumi AWS Nitric \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0645\u06cc\u200c\u06a9\u0646\u06cc\u0645\u060c \u0645\u0637\u0645\u0626\u0646 \u0634\u0648\u06cc\u062f \u06a9\u0647 \u0628\u0631\u0627\u06cc \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u0622\u0646 \u062a\u0646\u0638\u06cc\u0645 \u0634\u062f\u0647\u200c\u0627\u06cc\u062f. \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0628\u06cc\u0634\u062a\u0631\u06cc \u062f\u0631 \u0645\u0648\u0631\u062f \u0646\u062d\u0648\u0647 \u0631\u0627\u0647\u200c\u0627\u0646\u062f\u0627\u0632\u06cc \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 AWS \u062f\u0631 \u0627\u0633\u0646\u0627\u062f \u0627\u0631\u0627\u0626\u0647\u200c\u062f\u0647\u0646\u062f\u0647 Nitric AWS \u0628\u06cc\u0627\u0628\u06cc\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0645\u06cc \u062e\u0648\u0627\u0647\u06cc\u062f \u0628\u0627 Terraform \u06cc\u0627 \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u0647 \u0627\u0628\u0631 \u062f\u06cc\u06af\u0631\u06cc \u0645\u0633\u062a\u0642\u0631 \u0634\u0648\u06cc\u062f\u060c \u0627\u06cc\u0646 \u0646\u06cc\u0632 \u0627\u0645\u06a9\u0627\u0646 \u067e\u0630\u06cc\u0631 \u0627\u0633\u062a. \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0637\u0644\u0627\u0639\u0627\u062a \u0628\u06cc\u0634\u062a\u0631\u06cc \u062f\u0631 \u0645\u0648\u0631\u062f \u0646\u062d\u0648\u0647 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 Nitric \u062f\u0631 \u0633\u0627\u06cc\u0631 \u067e\u0644\u062a\u0641\u0631\u0645 \u0647\u0627 \u062f\u0631 \u0627\u0633\u0646\u0627\u062f \u0627\u0631\u0627\u0626\u0647 \u062f\u0647\u0646\u062f\u06af\u0627\u0646 Nitric \u067e\u06cc\u062f\u0627 \u06a9\u0646\u06cc\u062f.<\/p>\n<p>\u0633\u067e\u0633 \u0645\u06cc \u062a\u0648\u0627\u0646\u06cc\u062f \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 \u062f\u0633\u062a\u0648\u0631 \u0632\u06cc\u0631 \u0645\u0633\u062a\u0642\u0631 \u0634\u0648\u06cc\u062f:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>nitric up\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0628\u0647 \u0646\u0634\u0627\u0646\u06cc \u0648\u0628 \u0646\u0642\u0637\u0647 \u067e\u0627\u06cc\u0627\u0646\u06cc API \u06a9\u0647 \u067e\u0633 \u0627\u0632 \u062a\u06a9\u0645\u06cc\u0644 \u0627\u0633\u062a\u0642\u0631\u0627\u0631 \u062e\u0631\u0648\u062c\u06cc \u0645\u06cc\u200c\u0634\u0648\u062f\u060c \u062a\u0648\u062c\u0647 \u062f\u0627\u0634\u062a\u0647 \u0628\u0627\u0634\u06cc\u062f.<\/p>\n<p>\u0627\u06af\u0631 \u0628\u0639\u062f\u0627\u064b \u067e\u0631\u0648\u0698\u0647 \u0631\u0627 \u062a\u0645\u0627\u0645 \u06a9\u0631\u062f\u06cc\u062f\u060c \u0622\u0646 \u0631\u0627 \u062e\u0631\u0627\u0628 \u06a9\u0646\u06cc\u062f <code>nitric down<\/code>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"%D8%AA%D8%B3%D8%AA_%D8%B1%D9%88%DB%8C_AWS\"><\/span>\n<p>  \u062a\u0633\u062a \u0631\u0648\u06cc AWS<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0628\u0631\u0627\u06cc \u0622\u0632\u0645\u0627\u06cc\u0634 \u0627\u06cc\u0646 \u0633\u0631\u0648\u06cc\u0633\u060c \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u0632 \u0647\u0631 \u0627\u0628\u0632\u0627\u0631 \u062a\u0633\u062a API \u06a9\u0647 \u062f\u0648\u0633\u062a \u062f\u0627\u0631\u06cc\u062f\u060c \u0645\u0627\u0646\u0646\u062f cURL\u060c Postman \u0648 \u063a\u06cc\u0631\u0647 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u06a9\u0646\u06cc\u062f. \u062f\u0631 \u0627\u06cc\u0646\u062c\u0627 \u06cc\u06a9 \u0645\u062b\u0627\u0644 \u0628\u0627 \u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0627\u0632 cURL \u0622\u0648\u0631\u062f\u0647 \u0634\u062f\u0647 \u0627\u0633\u062a:<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> POST <span class=\"o\">{<\/span>your endpoint URL here<span class=\"o\">}<\/span>\/prompt <span class=\"nt\">-d<\/span> <span class=\"s2\">\"Hello, how are you?\"<\/span>\n<\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<h3><span class=\"ez-toc-section\" id=\"%D9%BE%D8%A7%D8%B3%D8%AE_%D9%86%D9%85%D9%88%D9%86%D9%87\"><\/span>\n<p>  \u067e\u0627\u0633\u062e \u0646\u0645\u0648\u0646\u0647<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>\u067e\u0627\u0633\u062e \u0634\u0627\u0645\u0644 \u0646\u062a\u0627\u06cc\u062c\u060c \u0628\u0647 \u0639\u0644\u0627\u0648\u0647 \u0633\u0627\u06cc\u0631 \u0627\u0628\u0631\u062f\u0627\u062f\u0647 \u0647\u0627 \u062e\u0648\u0627\u0647\u062f \u0628\u0648\u062f. \u062e\u0631\u0648\u062c\u06cc \u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646 \u062f\u0631 \u0622\u0631\u0627\u06cc\u0647 \u0627\u0646\u062a\u062e\u0627\u0628 \u06cc\u0627\u0641\u062a.<\/p>\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"p\">{<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"id\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"cmpl-61064b38-45f9-496d-86d6-fdae4bc3db97\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"object\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"text_completion\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"created\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">1729655327<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"model\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\".\/models\/Llama-3.2-1B-Instruct-Q4_K_M.gguf\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"choices\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">[<\/span><span class=\"w\">\n    <\/span><span class=\"p\">{<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"text\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">I'm doing well, thank you for asking. I'm here and ready to assist you, so that's a good start! How can I help you today?<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"index\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"logprobs\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span><span class=\"w\">\n      <\/span><span class=\"nl\">\"finish_reason\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"stop\"<\/span><span class=\"w\">\n    <\/span><span class=\"p\">}<\/span><span class=\"w\">\n  <\/span><span class=\"p\">],<\/span><span class=\"w\">\n  <\/span><span class=\"nl\">\"usage\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"prompt_tokens\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">26<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"completion_tokens\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">33<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"total_tokens\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"mi\">59<\/span><span class=\"w\">\n  <\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n<div class=\"highlight__panel js-actions-panel\">\n<div class=\"highlight__panel-action js-fullscreen-code-action\">\n    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-on\"><title>\u0648\u0627\u0631\u062f \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M16 3h6v6h-2V5h-4V3zM2 3h6v2H4v4H2V3zm18 16v-4h2v6h-6v-2h4zM4 19h4v2H2v-6h2v4z\"\/>\n<\/svg><\/p>\n<p>    <svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"20px\" height=\"20px\" viewbox=\"0 0 24 24\" class=\"highlight-action crayons-icon highlight-action--fullscreen-off\"><title>\u0627\u0632 \u062d\u0627\u0644\u062a \u062a\u0645\u0627\u0645 \u0635\u0641\u062d\u0647 \u062e\u0627\u0631\u062c \u0634\u0648\u06cc\u062f<\/title>\n    <path d=\"M18 7h4v2h-6V3h2v4zM8 9H2V7h4V3h2v6zm10 8v4h-2v-6h6v2h-4zM8 15v6H6v-4H2v-2h6z\"\/>\n<\/svg><\/p>\n<\/div>\n<\/div>\n<\/div>\n<h2><span class=\"ez-toc-section\" id=\"%D8%AE%D9%84%D8%A7%D8%B5%D9%87\"><\/span>\n<p>  \u062e\u0644\u0627\u0635\u0647<br \/>\n<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>\u0647\u0645\u0627\u0646\u0637\u0648\u0631 \u06a9\u0647 \u062f\u0631 \u0645\u062b\u0627\u0644 \u06a9\u062f \u0645\u0634\u0627\u0647\u062f\u0647 \u06a9\u0631\u062f\u06cc\u062f\u060c \u0645\u0627 \u06cc\u06a9 \u0633\u0627\u062e\u062a\u0627\u0631 \u0627\u0639\u0644\u0627\u0646 \u0646\u0633\u0628\u062a\u0627\u064b \u0627\u0628\u062a\u062f\u0627\u06cc\u06cc \u062a\u0646\u0638\u06cc\u0645 \u06a9\u0631\u062f\u0647\u200c\u0627\u06cc\u0645\u060c \u0627\u0645\u0627 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0627\u06cc\u0646 \u0633\u0627\u062e\u062a\u0627\u0631 \u0631\u0627 \u06af\u0633\u062a\u0631\u0634 \u062f\u0647\u06cc\u062f \u062a\u0627 \u0627\u0639\u0644\u0627\u0646\u200c\u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631\u06cc \u0631\u0627 \u0634\u0627\u0645\u0644 \u0634\u0648\u062f\u060c \u0627\u0632 \u062c\u0645\u0644\u0647 \u0627\u0639\u0644\u0627\u0646\u200c\u0647\u0627\u06cc \u0633\u06cc\u0633\u062a\u0645\u06cc \u06a9\u0647 \u0628\u0647 \u0645\u062d\u062f\u0648\u062f \u06a9\u0631\u062f\u0646\/\u0647\u062f\u0627\u06cc\u062a \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0645\u062f\u0644 \u06cc\u0627 \u062d\u062a\u06cc \u062a\u0639\u0627\u0645\u0644\u0627\u062a \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631 \u0628\u0627 \u0645\u062f\u0644 \u0647\u0645\u0686\u0646\u06cc\u0646\u060c \u062f\u0631 \u0627\u06cc\u0646 \u0645\u062b\u0627\u0644\u060c \u0645\u0627 \u0645\u062f\u0644 \u0631\u0627 \u0645\u0633\u062a\u0642\u06cc\u0645\u0627\u064b \u0628\u0647\u200c\u0639\u0646\u0648\u0627\u0646 \u06cc\u06a9 API \u062f\u0631 \u0645\u0639\u0631\u0636 \u062f\u06cc\u062f \u0642\u0631\u0627\u0631 \u0645\u06cc\u200c\u062f\u0647\u06cc\u0645\u060c \u0627\u0645\u0627 \u0627\u06cc\u0646 \u0632\u0645\u0627\u0646 \u067e\u0627\u0633\u062e\u06af\u0648\u06cc\u06cc \u0631\u0627 \u0628\u0647 30 \u062b\u0627\u0646\u06cc\u0647 \u062f\u0631 AWS \u0628\u0627 API Gateway \u0645\u062d\u062f\u0648\u062f \u0645\u06cc\u200c\u06a9\u0646\u062f.<\/p>\n<p>\u062f\u0631 \u0631\u0627\u0647\u0646\u0645\u0627\u0647\u0627\u06cc \u0622\u06cc\u0646\u062f\u0647\u060c \u0646\u0634\u0627\u0646 \u062e\u0648\u0627\u0647\u06cc\u0645 \u062f\u0627\u062f \u06a9\u0647 \u0686\u06af\u0648\u0646\u0647 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u062f \u0641\u0631\u0627\u062a\u0631 \u0627\u0632 \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0633\u0627\u062f\u0647 \u06cc\u06a9\u200c\u0628\u0627\u0631\u0647 \u0628\u0647 \u062a\u0639\u0627\u0645\u0644\u200c\u0647\u0627\u06cc \u067e\u06cc\u0686\u06cc\u062f\u0647\u200c\u062a\u0631\u060c \u0645\u0627\u0646\u0646\u062f \u062d\u0641\u0638 \u0628\u0627\u0641\u062a \u0628\u06cc\u0646 \u062f\u0631\u062e\u0648\u0627\u0633\u062a\u200c\u0647\u0627 \u0628\u0631\u0648\u06cc\u062f. \u0645\u0627 \u0647\u0645\u0686\u0646\u06cc\u0646 \u0645\u06cc\u200c\u062a\u0648\u0627\u0646\u06cc\u0645 Websockets \u0648 \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u062c\u0631\u06cc\u0627\u0646\u06cc \u0631\u0627 \u0628\u0631\u0627\u06cc \u0627\u0631\u0627\u0626\u0647 \u062a\u062c\u0631\u0628\u0647 \u06a9\u0627\u0631\u0628\u0631\u06cc \u0628\u0647\u062a\u0631 \u0628\u0631\u0627\u06cc \u067e\u0627\u0633\u062e\u200c\u0647\u0627\u06cc \u0628\u0632\u0631\u06af\u200c\u062a\u0631 \u0627\u0636\u0627\u0641\u0647 \u06a9\u0646\u06cc\u0645.<\/p>\n<\/p><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Summarize this content to 400 words in Persian Lang Llama 3.2 1B \u06cc\u06a9 \u0645\u062f\u0644 \u0647\u0648\u0634 \u0645\u0635\u0646\u0648\u0639\u06cc \u0633\u0628\u06a9 \u0648\u0632\u0646 \u0627\u0633\u062a \u06a9\u0647 \u0622\u0646 \u0631\u0627 \u0628\u0631\u0627\u06cc \u0628\u0631\u0646\u0627\u0645\u0647 \u0647\u0627\u06cc \u0628\u062f\u0648\u0646 \u0633\u0631\u0648\u0631 \u062c\u0627\u0644\u0628 \u0645\u06cc \u06a9\u0646\u062f \u0632\u06cc\u0631\u0627 \u0645\u06cc \u062a\u0648\u0627\u0646 \u0622\u0646 \u0631\u0627 \u0646\u0633\u0628\u062a\u0627\u064b \u0633\u0631\u06cc\u0639 \u0628\u062f\u0648\u0646 \u0646\u06cc\u0627\u0632 \u0628\u0647 \u0634\u062a\u0627\u0628 GPU \u0627\u062c\u0631\u0627 \u06a9\u0631\u062f. \u0645\u0627 \u0627\u0632 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc\u06cc \u0627\u0632 Hugging Face \u0648 Nitric \u0628\u0631\u0627\u06cc \u0646\u0634\u0627\u0646 \u062f\u0627\u062f\u0646 &hellip;<\/p>\n","protected":false},"author":2,"featured_media":85413,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"fifu_image_url":"","fifu_image_alt":"","footnotes":""},"categories":[339],"tags":[],"class_list":["post-85412","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dev"],"_links":{"self":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/85412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/comments?post=85412"}],"version-history":[{"count":0,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/posts\/85412\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media\/85413"}],"wp:attachment":[{"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/media?parent=85412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/categories?post=85412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nabfollower.com\/blog\/wp-json\/wp\/v2\/tags?post=85412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}