deploy: run the MCP server in k3s under the paper-doc namespace

The server has to be reachable by the Harness, Claude Code and Codex, and all
three of those are clients of an address rather than of a source tree — a
stdio child process only serves whichever machine spawned it. So it is packaged
and deployed: namespace `paper-doc`, one Deployment, one NodePort Service at
`http://192.168.1.88:30931/mcp` behind a bearer token, talking to the
in-cluster TiDB Service rather than the node's NodePort.

The namespace is `paper-doc` and not `paper_doc`, which is not a preference:
a namespace name is an RFC 1123 label, underscores are outside that alphabet,
and the API server rejects it outright. The database keeps its underscore.

Two routes were added to the server for the deployment, and they are the only
unauthenticated ones. `/healthz` answers "is this process alive" without
touching anything, `/readyz` answers "can it serve" by running `SELECT 1`. The
split is the point: a failing readiness probe takes the pod out of the
Service's endpoints and puts it back when TiDB returns, while a failing
liveness probe kills a process that was working correctly.

There is no container runtime on this host — k3s runs containerd and neither
`ctr` nor `crictl` builds from a Dockerfile — so `deploy/build-mcp-image.sh`
builds inside a throwaway privileged docker-in-docker pod, mounting the working
tree through a hostPath so the build context is the tree you are looking at,
uncommitted edits included. It reads the registry credentials out of the
cluster's own pull secret, so no password is in the script or in a shell
history. One Dockerfile serves both entry points, because the API and the MCP
server share the domain layer and differ by one argv.

Verified against the deployed Service, not against a local process: 47 MCP
checks pass over the NodePort, and the Node MCP SDK — the one the Harness's
mcp-client plugin uses — lists all 29 tools and calls them. The same 47 pass in
stdio and against a local HTTP server, and the REST suite still passes its 45.
This commit is contained in:
2026-09-21 00:08:17 +08:00
parent 5a6461eeec
commit 7e7820f823
10 changed files with 735 additions and 14 deletions
+142
View File
@@ -0,0 +1,142 @@
# 部署到 k3s
paper-doc 的 MCP 服务跑在 k3s 里,命名空间 **`paper-doc`**,客户端通过
`http://192.168.1.88:30931/mcp` 连接。
> **为什么不是 `paper_doc`**:Kubernetes 的命名空间名必须是 RFC 1123 label,
> 下划线不在允许的字符集里(`[a-z0-9]([-a-z0-9]*[a-z0-9])?`),API Server 会直接拒绝。
> 所以命名空间用连字符 `paper-doc`;数据库名和表名仍然是 `paper_doc`,两者不冲突。
## 部署了什么
| 资源 | 名字 | 说明 |
|---|---|---|
| Namespace | `paper-doc` | `deploy/k8s/00-namespace.yaml` |
| Deployment | `paper-doc-mcp` | 1 副本,跑 MCP 服务的 http 传输 |
| Service | `paper-doc-mcp` | NodePort **30931** → 容器 8931 |
| ConfigMap | `paper-doc-config` | 非敏感环境变量(数据库地址、监听配置) |
| Secret | `paper-doc-secrets` | `DB_USER` / `DB_PASSWORD` / `MCP_HTTP_TOKEN` |
| Secret | `aliyun-acr` | 镜像仓库拉取凭证(从 `default` 复制) |
镜像:`registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:v2`
**数据库**:TiDB 已经在这个集群里,Deployment 用集群内的 Service
`tidb-tidb.tidb:4000`,而不是宿主机用的 NodePort `192.168.1.88:32738`。
两者是同一份数据;集群内用 Service 的好处是节点地址变了也不用改配置。
## 访问地址
| 从哪里连 | 地址 |
|---|---|
| 宿主机 / 局域网 | `http://192.168.1.88:30931/mcp` |
| 集群内 Pod | `http://paper-doc-mcp.paper-doc.svc.cluster.local:8931/mcp` |
| 健康检查(免鉴权) | `http://192.168.1.88:30931/healthz`、`/readyz` |
除 `/healthz` 和 `/readyz` 外,所有请求都要带
`Authorization: Bearer <MCP_HTTP_TOKEN>`,否则返回 `401`。
Token 在 Secret 里,取法:
```bash
export KUBECONFIG=$HOME/.kube/config
kubectl -n paper-doc get secret paper-doc-secrets \
-o jsonpath='{.data.MCP_HTTP_TOKEN}' | base64 -d; echo
```
## 日常操作
先记住这一句,否则 `kubectl` 会去读 root-only 的
`/etc/rancher/k3s/k3s.yaml` 并报 permission denied:
```bash
export KUBECONFIG=$HOME/.kube/config
```
```bash
# 状态
kubectl -n paper-doc get deploy,svc,pods
kubectl -n paper-doc logs deploy/paper-doc-mcp --tail=50
# 重新构建并推送镜像(需要一个 tag,见下面「两个坑」)
deploy/build-mcp-image.sh v3
# 发布新版本
kubectl -n paper-doc set image deployment/paper-doc-mcp \
mcp=registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:v3
kubectl -n paper-doc rollout status deployment/paper-doc-mcp
# 回滚
kubectl -n paper-doc rollout undo deployment/paper-doc-mcp
# 改配置 / 换 token
kubectl -n paper-doc edit configmap paper-doc-config
kubectl -n paper-doc create secret generic paper-doc-secrets \
--from-literal=DB_USER='Govin' \
--from-literal=DB_PASSWORD='<tidb-password>' \
--from-literal=MCP_HTTP_TOKEN="$(openssl rand -hex 32)" \
--dry-run=client -o yaml | kubectl apply -f -
kubectl -n paper-doc rollout restart deployment/paper-doc-mcp
```
**验证部署**——不是看 Pod 是不是 Running,而是真的用 MCP 客户端跑一遍:
```bash
cd backend
.venv/bin/python scripts/smoke_mcp.py \
--url http://192.168.1.88:30931/mcp --token "$(cat /tmp/paper-mcp-token)"
# 47 项检查全过 = 集群里这套东西真的能写论文
```
## 构建:为什么是 DinD Pod
这台宿主机上没有容器运行时——k3s 用的是 containerd,节点上的 `ctr` / `crictl`
都是 k3s 的软链,谁都不能从 Dockerfile 构建。所以构建放在集群里一个临时的
docker-in-docker Pod 里做,源码通过 hostPath 挂进去(`/workspace`),
这样构建上下文就是当前工作区,未提交的改动也在里面。
`deploy/build-mcp-image.sh` 把这一整套包好了:起 Pod → 登录 → 构建 → 推送 → 删 Pod
(`trap` 保证失败也会删)。**仓库凭证是从集群里已有的 `aliyun-acr-secret` 读出来的**,
所以脚本里没有任何密码,也不会进 shell history。
## 两个坑(都踩过了,别再踩)
**1. 镜像 tag 复用会发布不出新代码。**
Deployment 的 `imagePullPolicy: IfNotPresent`,节点按 tag 缓存镜像。
用同样的 `:v1` 重新构建推送后,Pod 重启拿到的还是旧镜像——
现象是新 Pod 起来了、但行为跟没改一样。**每次构建换一个 tag**(v2、v3…),
或者临时把 `imagePullPolicy` 改成 `Always`(代价是每次启动都要连仓库)。
**2. 源码权限 0600 会让容器读不到代码。**
工作区的文件是 `rw-------`,`COPY` 会把权限一起带进镜像,
于是非 root(uid 10001)的容器启动即
`can't open file '/app/scripts/mcp_server.py': [Errno 13] Permission denied`,
然后 CrashLoopBackOff。Dockerfile 里 `RUN chmod -R a+rX /app` 解决——
镜像里的东西只需要可读,不需要任何人可写。
## 探针:为什么是两个
| 探针 | 路径 | 语义 |
|---|---|---|
| liveness | `/healthz` | 进程还活着吗——不碰数据库,所以数据库抖一下不会变成反复重启 |
| readiness | `/readyz` | 现在能服务吗——执行 `SELECT 1`,TiDB 不通就把 Pod 摘出 Service,通了自动加回来 |
失败 readiness 只是把 Pod 从负载均衡里摘掉,失败 liveness 会杀掉进程。
数据库短暂不可用只应该是前者,所以这个分工是刻意的。
## 还没有部署的
- **REST API**:同一个镜像,换 command 即可(`uvicorn app.main:app --host 0.0.0.0 --port 8000`),
再加一个 init 容器跑 `alembic upgrade head`。配置模板写在
`deploy/k8s/10-paper-doc-mcp.yaml` 末尾的注释里,解开注释即可。
- **前端 SPA**:需要额外构建(pnpm build)+ nginx 镜像,目前没做。
## 文件清单
```
deploy/
├── build-mcp-image.sh # DinD 构建并推送镜像
├── README.md # 本文件
└── k8s/
├── 00-namespace.yaml
├── 10-paper-doc-mcp.yaml # ConfigMap + Deployment + Service(含 API 的注释模板)
└── secret.example.yaml # Secret 模板,真值不入库
```
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env bash
# Build the paper-doc image and push it to the registry the cluster pulls from.
#
# Why a pod builds it
# -------------------
# There is no container runtime on this host: k3s runs containerd, and the
# binaries on the node are the k3s ones (`ctr`, `crictl`), neither of which
# builds from a Dockerfile. So the build runs where a builder already exists —
# a privileged docker-in-docker pod inside the cluster — and the source reaches
# it through a hostPath mount rather than `kubectl cp`, which keeps the build
# context exactly the working tree you are looking at, uncommitted edits
# included.
#
# The credentials are read out of the cluster's own pull secret rather than
# taken as arguments, so this script has none in it and nothing is pasted into a
# shell history.
#
# Usage (from anywhere):
#
# deploy/build-mcp-image.sh # tag :v1
# deploy/build-mcp-image.sh v2 # tag :v2
# IMAGE=... deploy/build-mcp-image.sh # a different repository entirely
#
# Prerequisite: `export KUBECONFIG=$HOME/.kube/config` (the k3s binary defaults
# to /etc/rancher/k3s/k3s.yaml, which is root-only on this host).
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
TAG="${1:-v1}"
IMAGE="${IMAGE:-registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:${TAG}}"
BUILD_NAMESPACE="${BUILD_NAMESPACE:-default}"
BUILD_POD="${BUILD_POD:-dind-build-paper-doc}"
REGISTRY_HOST="${IMAGE%%/*}"
# The old build pod is what proved this image is already on the node; the same
# one is used here so a rebuild does not have to reach docker.io, which this
# network cannot.
BUILDER_IMAGE="${BUILDER_IMAGE:-docker:27-dind}"
export KUBECONFIG="${KUBECONFIG:-$HOME/.kube/config}"
log() { printf '\n=== %s\n' "$*"; }
cleanup() {
log "removing the build pod"
kubectl -n "$BUILD_NAMESPACE" delete pod "$BUILD_POD" --ignore-not-found --wait=false >/dev/null 2>&1 || true
}
trap cleanup EXIT
log "build context: $REPO_ROOT/backend"
log "image: $IMAGE"
kubectl -n "$BUILD_NAMESPACE" delete pod "$BUILD_POD" --ignore-not-found >/dev/null 2>&1 || true
log "starting the docker-in-docker builder"
# `DOCKER_TLS_CERTDIR=""` keeps the daemon on tcp://localhost:2375 inside the
# pod, so `docker` in the same container needs no certificate dance. The pod is
# privileged because that is what a docker daemon needs; it lives for the length
# of one build and is deleted by the trap.
cat <<YAML | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: ${BUILD_POD}
namespace: ${BUILD_NAMESPACE}
labels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: image-build
spec:
restartPolicy: Never
containers:
- name: dind
image: ${BUILDER_IMAGE}
securityContext:
privileged: true
env:
- name: DOCKER_TLS_CERTDIR
value: ""
volumeMounts:
- name: workspace
mountPath: /workspace
- name: docker-storage
mountPath: /var/lib/docker
volumes:
- name: workspace
hostPath:
path: ${REPO_ROOT}
type: Directory
- name: docker-storage
emptyDir: {}
YAML
log "waiting for the daemon"
kubectl -n "$BUILD_NAMESPACE" wait --for=condition=Ready "pod/$BUILD_POD" --timeout=180s
for _ in $(seq 1 30); do
if kubectl -n "$BUILD_NAMESPACE" exec "$BUILD_POD" -- docker info >/dev/null 2>&1; then
break
fi
sleep 2
done
kubectl -n "$BUILD_NAMESPACE" exec "$BUILD_POD" -- docker info --format 'daemon ok: {{.ServerVersion}}'
log "logging in to $REGISTRY_HOST"
# Read out of the pull secret the cluster already uses: the push credential and
# the pull credential are the same account, and a second copy of it in a script
# is a second place it can go stale.
kubectl -n default get secret aliyun-acr-secret -o jsonpath='{.data.\.dockerconfigjson}' \
| base64 -d \
| python3 -c '
import json, sys
auth = next(iter(json.load(sys.stdin)["auths"].values()))
print(auth["username"], auth["password"])
' \
| kubectl -n "$BUILD_NAMESPACE" exec -i "$BUILD_POD" -- sh -c \
'read -r user pass; printf "%s" "$pass" | docker login -u "$user" --password-stdin "$0"' "$REGISTRY_HOST"
log "building"
# --network=host: the build needs the same mirror access the node has for pip,
# and the pod's own network namespace would otherwise be a second, untested path
# to the internet.
kubectl -n "$BUILD_NAMESPACE" exec "$BUILD_POD" -- \
docker build --network=host --tag "$IMAGE" /workspace/backend
log "pushing"
kubectl -n "$BUILD_NAMESPACE" exec "$BUILD_POD" -- docker push "$IMAGE"
log "done: $IMAGE"
cat <<EOF
Next:
kubectl -n paper_doc set image deployment/paper-doc-mcp mcp=$IMAGE
kubectl -n paper_doc rollout status deployment/paper-doc-mcp
EOF
+18
View File
@@ -0,0 +1,18 @@
# The namespace the whole paper-doc deployment lives in.
#
# It is created here rather than by `kubectl create ns` so that a rebuild of the
# cluster from these files brings the application up in the same shape it has
# now — the namespace name is part of the deployment, not an accident of how it
# was first applied.
#
# The name is `paper-doc`, not `paper_doc`: a namespace is an RFC 1123 label and
# underscores are not in that alphabet, so the API server rejects the name the
# project uses for its database and its tables. Every other resource in this
# namespace is hyphenated for the same reason.
apiVersion: v1
kind: Namespace
metadata:
name: paper-doc
labels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/part-of: paper-doc
+192
View File
@@ -0,0 +1,192 @@
# paper-doc MCP server: ConfigMap, Deployment, Service.
#
# The REST API and the MCP server are the same image with different commands.
# Only the MCP server is deployed here, because that is what a client connects
# to; the API is described in a commented block at the end of this file rather
# than deployed, so that turning it on is a decision rather than a surprise.
#
# The database is TiDB, already running in this cluster as `tidb-tidb.tidb:4000`
# — the in-cluster Service, not the 32738 NodePort the host uses. Both reach the
# same rows; the Service is the one that survives the node's address changing.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: paper-doc-config
namespace: paper-doc
labels:
app.kubernetes.io/name: paper-doc
data:
DB_HOST: tidb-tidb.tidb
DB_PORT: "4000"
DB_NAME: paper_doc
APP_NAME: paper-doc API
DEBUG: "false"
# The container's own defaults, restated here because the Deployment is where
# a reader looks for them.
MCP_TRANSPORT: http
MCP_HOST: 0.0.0.0
MCP_PORT: "8931"
MCP_PATH: /mcp
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: paper-doc-mcp
namespace: paper-doc
labels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: mcp
spec:
# One replica. The server is stateless — every tool opens its own database
# session and closes it — so scaling is safe, but a second replica would only
# add connections to TiDB without adding throughput a model can use.
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: mcp
template:
metadata:
labels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: mcp
spec:
imagePullSecrets:
- name: aliyun-acr
securityContext:
runAsNonRoot: true
runAsUser: 10001
fsGroup: 10001
seccompProfile:
type: RuntimeDefault
containers:
- name: mcp
image: registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:v2
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 8931
envFrom:
- configMapRef:
name: paper-doc-config
env:
- name: DB_USER
valueFrom:
secretKeyRef:
name: paper-doc-secrets
key: DB_USER
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: paper-doc-secrets
key: DB_PASSWORD
# Without it the HTTP transport is unauthenticated. The server warns
# on stderr when it binds a non-loopback address and finds no token,
# which is the only signal a reader gets that this key is missing.
- name: MCP_HTTP_TOKEN
valueFrom:
secretKeyRef:
name: paper-doc-secrets
key: MCP_HTTP_TOKEN
# Readiness asks the database; liveness does not. A TiDB blip should
# take the pod out of the Service's endpoints and put it back when the
# blip ends — not restart a process that is working correctly.
readinessProbe:
httpGet:
path: /readyz
port: http
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
livenessProbe:
httpGet:
path: /healthz
port: http
initialDelaySeconds: 10
periodSeconds: 30
timeoutSeconds: 5
failureThreshold: 3
resources:
requests:
cpu: 50m
memory: 128Mi
limits:
cpu: 1000m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
volumeMounts:
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: paper-doc-mcp
namespace: paper-doc
labels:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: mcp
spec:
# NodePort, not ClusterIP: the clients that matter here are not in the
# cluster. The Harness, Claude Code and Codex all run on the node or on a
# machine on the LAN, and `http://192.168.1.88:30931/mcp` is the same address
# from both. In-cluster callers use the ClusterIP on port 8931.
type: NodePort
selector:
app.kubernetes.io/name: paper-doc
app.kubernetes.io/component: mcp
ports:
- name: http
port: 8931
targetPort: http
nodePort: 30931
# --- the REST API, if you want it in the cluster too -------------------------
#
# Same image, different command; it needs no MCP_HTTP_TOKEN and answers on 8000.
# The schema has to exist before the API serves it, which is what the init
# container is for:
#
# ---
# apiVersion: apps/v1
# kind: Deployment
# metadata:
# name: paper-doc-api
# namespace: paper-doc
# spec:
# replicas: 1
# selector:
# matchLabels: {app.kubernetes.io/name: paper-doc, app.kubernetes.io/component: api}
# template:
# metadata:
# labels: {app.kubernetes.io/name: paper-doc, app.kubernetes.io/component: api}
# spec:
# imagePullSecrets: [{name: aliyun-acr}]
# initContainers:
# - name: migrate
# image: registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:v2
# command: ["alembic", "upgrade", "head"]
# envFrom: [{configMapRef: {name: paper-doc-config}}]
# env: [...] # the same three secret keys as above
# containers:
# - name: api
# image: registry.cn-hangzhou.aliyuncs.com/mcp_skill/paper-doc-mcp:v2
# command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
# ports: [{name: http, containerPort: 8000}]
# envFrom: [{configMapRef: {name: paper-doc-config}}]
# env: [...]
# readinessProbe: {httpGet: {path: /api/health, port: http}}
+25
View File
@@ -0,0 +1,25 @@
# Template for the paper-doc Secret. Do not fill this file in and commit it —
# create the real Secret out of band, so the values live in the cluster and not
# in git history:
#
# kubectl -n paper-doc create secret generic paper-doc-secrets \
# --from-literal=DB_USER='Govin' \
# --from-literal=DB_PASSWORD='<tidb-password>' \
# --from-literal=MCP_HTTP_TOKEN="$(openssl rand -hex 32)"
#
# Rotating the token is then one `kubectl create secret ... --dry-run=client -o
# yaml | kubectl apply -f -` plus a rollout restart, and no image rebuild.
#
# The registry pull secret is separate and not described here: it is
# `aliyun-acr`, copied into this namespace from `default`, and it holds the
# container registry's credentials rather than the application's.
apiVersion: v1
kind: Secret
metadata:
name: paper-doc-secrets
namespace: paper-doc
type: Opaque
stringData:
DB_USER: Govin
DB_PASSWORD: change-me
MCP_HTTP_TOKEN: change-me