[TOC] ## ollama 1. 可支持圖片解析 **應用場景** * 開發輔助 * 代碼補全 * 代碼審查 * Bug 修復建議 * 本地知識庫 * 文檔問答 * 技術資料查詢 * 學習輔助 **離線AI 助手** * 文本生成 * 內容創作 * 數據分析 **優勢** * 性能優勢 * 本地運行，響應快速 * 支持 GPU 加速 * 資源占用可控 * 隱私安全 * 數據不出本地 * 適合處理敏感信息 * 符合數據合規要求 * 成本效益 * 一次部署，永久使用 * 無需支付 API 費用 * 可控的硬件投入 ## 安裝 https://ollama.com/download/ 下載對應系統的版本 ### 指定模型值 ``` export OLLAMA_MODELS=xxx ``` ## 命令 ``` Usage: ollama [flags] ollama [command] Available Commands: serve Start ollama create Create a model from a Modelfile show Show information for a model run Run a model pull Pull a model from a registry push Push a model to a registry list List models cp Copy a model rm Remove a model help Help about any command Flags: -h, --help help for ollama -v, --version Show version information Use "ollama [command] --help" for more information about a command. ``` ## 接口文檔 * [Generate a completion](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion) * [Generate a chat completion](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion) * [Create a Model](https://github.com/ollama/ollama/blob/main/docs/api.md#create-a-model) * [List Local Models](https://github.com/ollama/ollama/blob/main/docs/api.md#list-local-models) * [Show Model Information](https://github.com/ollama/ollama/blob/main/docs/api.md#show-model-information) * [Copy a Model](https://github.com/ollama/ollama/blob/main/docs/api.md#copy-a-model) * [Delete a Model](https://github.com/ollama/ollama/blob/main/docs/api.md#delete-a-model) * [Pull a Model](https://github.com/ollama/ollama/blob/main/docs/api.md#pull-a-model) * [Push a Model](https://github.com/ollama/ollama/blob/main/docs/api.md#push-a-model) * [Generate Embeddings](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings) ## 示例 ### 命令行啟動 部署后,就可以在命令行或api 中調用 ``` >>> """Hello, ... world! ... """ I'm a basic program that prints the famous "Hello, world!" message to the console. ``` 啟動服務后,可在 調用web 接口 1. 生成文本 ``` curl http://localhost:11434/api/generate -d '{ "model": "llama3", "prompt":"Why is the sky blue?" }' ``` 2.生成對話 ``` curl http://localhost:11434/api/chat -d '{ "model": "llama3", "messages": [ { "role": "user", "content": "why is the sky blue?" } ] }' ``` ### GPU 1. 下載 [cuda ](https://developer.nvidia.com/cuda-downloads) 2. 可先通過 `ollama run 模型` 看 gpu顯卡內存是否上漲 3. 重啟ollama,ollama 服務需要api 接口調用時,加載到gpu 顯卡中 ``` curl -X POST http://localhost:11434/api/chat -d { "model": "llama2-chinese", "messages": [ { "role": "user", "content": "why is the sky blue?" }, { "role": "assistant", "content": "due to rayleigh scattering." }, { "role": "user", "content": "how is that different than mie scattering?" } ] } ```