安裝Docker
參考：http://www.lxweimin.com/p/62bac0a1060a
Docker支持Nvidia驅動
參考：docker支持nvidia gpu http://www.lxweimin.com/p/ed1e54c2a972
安裝Ollama
# 拉取鏡像
docker pull ollama/ollama:0.5.7
# Docker方式運行Ollama
mkdir -p /data/ollama
docker run -d --gpus=all -v /data/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:0.5.7
--gpus=all參數是為了在docker容器內可以使用GPU
--gpus "device=0": 使用第一個 GPU 設備--gpus "device=2,3": 使用第三和第四個 GPU 設備
由于習慣docker-compose，以docker-compose方式啟動的話，上述docker run可以用如下命令代替：
mkdir -p /data/ollama/data
cat > /data/ollama/docker-compose.yml <<"EOF"
services:
ollama:
image: ollama/ollama:0.5.7
container_name: ollama
restart: always
ports:
- "11434:11434"
volumes:
- /data/ollama/data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
# count 與 device_ids 二選一
# count指定需要使用的GPU數量；值為int數組型或all；
# device_ids: ["0", "3"]
count: "all"
capabilities: ["gpu"]
EOF
# 啟動Ollama
docker-compose up -d
Ollama更多環境變量，可以參考：https://zhuanlan.zhihu.com/p/712642806
運行Deepseek方式一：直接運行Ollama官方大模型
官方模型：https://ollama.com/search
# 進入容器內
docker exec -it ollama bash
# -------------以下命令都在容器內執行-------------
# 運行deepseek-r1:7b
ollama run deepseek-r1:7b
效果如下：
[root@bogon ollama]# docker exec -it ollama bash
root@e2809411e208:/# ollama run deepseek-r1:7b
pulling manifest
pulling 96c415656d37... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████ ▏ 4.7 GB/4.7 GB 14 MB/s 0s
pulling 369ca498f347... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 387 B
pulling 6e4c38e1172f... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 1.1 KB
pulling f4d24e9138dd... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 148 B
pulling 40fb844194b2... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 487 B
verifying sha256 digest
writing manifest
success
>>> 你是誰？
<think>
</think>
您好！我是由中國的深度求索（DeepSeek）公司開發的智能助手DeepSeek-R1。如您有任何任何問題，我會盡我所能為您提供幫助。
>>> Send a message (/? for help)
運行Deepseek方式二：通過離線gguf文件運行大模型
1. 下載gguf文件
文件來源：
下載方式：
- wget
wget -b https://www.modelscope.cn/models/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF/resolve/master/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf
- modelscope命令（推薦）
2. 編寫 modelfile
cat > /data/ollama/data/gguf/deepseek-r1-70B-modelfile <<"EOF"
FROM ./DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf
# 設置對話終止符
PARAMETER stop "<｜begin▁of▁sentence｜>"
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER stop "<｜User｜>"
PARAMETER stop "<｜Assistant｜>"
SYSTEM """
"""
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}"""
EOF
modelfile必須與gguf文件同目錄，否則需要修改modelfile第一行的相對位置
3. 創建模型
# 通過gguf創建模型(docker內執行）
ollama create deepseek-r1-custom:70B -f /root/.ollama/mygguf/deepseek-r1-70B-modelfile
4. 運行模型
# docker內執行
ollama run deepseek-r1-custom:70B
通過Ollama API 測試模型
# 通過ollama api測試模型
curl http://10.3.6.38:11434/api/chat -d '{
"model": "deepseek-r1-bidr:70b",
"messages": [
{
"role": "user",
"content": "為什么天空是藍色的?"
}
],
"stream": true
}'
效果：
[root@bogon ~]# curl http://10.3.6.38:11434/api/chat -d '{
"model": "deepseek-r1-bidr:70b",
"messages": [
{
"role": "user",
"content": "為什么天空是藍色的?"
}
],
"stream": true
}'
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.490403406Z","message":{"role":"assistant","content":"\u003cthink\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.62964314Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.782469849Z","message":{"role":"assistant","content":"\u003c/think\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.925424223Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.060309984Z","message":{"role":"assistant","content":"天"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.209522495Z","message":{"role":"assistant","content":"空"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.369515693Z","message":{"role":"assistant","content":"呈"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.52581505Z","message":{"role":"assistant","content":"現"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.686391838Z","message":{"role":"assistant","content":"藍"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.82169606Z","message":{"role":"assistant","content":"色"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.969790941Z","message":{"role":"assistant","content":"是"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.105679167Z","message":{"role":"assistant","content":"因為"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.243647488Z","message":{"role":"assistant","content":"當"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.380513238Z","message":{"role":"assistant","content":"太"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.525637676Z","message":{"role":"assistant","content":"陽"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.669904633Z","message":{"role":"assistant","content":"光"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.813863747Z","message":{"role":"assistant","content":"照"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.970479134Z","message":{"role":"assistant","content":"射"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.109944242Z","message":{"role":"assistant","content":"到"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.242174673Z","message":{"role":"assistant","content":"地球"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.381833905Z","message":{"role":"assistant","content":"大"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.51949058Z","message":{"role":"assistant","content":"氣"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.66205966Z","message":{"role":"assistant","content":"層"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.798480342Z","message":{"role":"assistant","content":"時"},"done":false}
