backend: let a model put a figure in the paper, and set a formula's alignment

Two things the editor gained and MCP could not do, both because the tool layer
had been told about the domain's columns by hand.

**A figure.** Bytes go to MinIO and the sentence keeps a Markdown address, so a
picture is ordinary text — which means the writing tools already carry it, and
the only half a model cannot do is the upload: it has bytes and no way to send
multipart. `image_upload` takes base64 and answers with the URL, the Markdown
and the *sentence object* the editor produces when a picture is dropped in, so a
figure is written by putting that object into `sentences` unchanged. It does not
fetch a URL the caller names: a service inside the cluster that will GET any
address it is handed is an SSRF primitive aimed at everything beside it.

The width argument is the reader's rule or it is refused. `{width=70%}` is
parsed by `frontend/src/utils/rich.ts`, and a value it cannot read is not an
error there — the picture just prints at its natural size. A size that silently
does not apply is worse than a refusal, so the syntax and the range (pixels
1–4000, per cent 1–100) are mirrored in `app/mcp/tools/images.py`, with the
usual comment naming the other side.

`paragraph_text` also had to learn what a figure is: a sentence holding nothing
but an image is a 插图 — the view prints it on its own line with the brackets'
text as its caption — and the text a tool hands back broke lines around a
display formula but not around a picture. Same rule as `figureOf`, same pair of
implementations, same warning about changing both.

**A formula's alignment.** `paper_sentence.formula_align` reached the editor and
the REST API and stopped there: `SentenceSpec` had no such field, so a model that
sent one got a successful call and a centred formula. The column is now declared,
carried through every write path and reported by every read path.

That is the second column lost the same way, so the fix is structural rather
than another two edits. `SENTENCE_FIELDS` is read from
`app.schemas.paper.SentenceInput` instead of written out, the payload is built
by one `sentence_payload()`, and the three append paths spread
`SentenceCreate(**item.model_dump())` rather than copying fields. One edit
remains — declaring the field in `specs.py` — and smoke_mcp.py's first step
compares the two field sets, so forgetting it now fails the suite instead of
losing the field. `sort` is declared too, which it never was: a model that sent
one had it dropped in silence.

smoke_mcp.py: 99 checks to 119. The figure step uploads a real 1x1 GIF (the same
bytes twice, and the key is the same object), writes the returned sentence as it
came back, checks the paragraph prints it alone on its line and that an image
inside a sentence does not break the line, and takes the four refusals — not a
picture, an unparseable size, an out-of-range one, not base64. The alignment step
writes a right-aligned formula, moves the sentence to left on its own, and takes
the paragraph to another position to prove the setting travels with it.

Deployed as v6, and the pod now carries MINIO_ENDPOINT (the in-cluster Service)
plus the two credential keys from the Secret; without them the image tool answers
with the missing variable names and every writing tool still works.
This commit is contained in:
2026-09-24 10:54:27 +08:00
parent eb0f12b8cf
commit ebaf9e558b
13 changed files with 816 additions and 79 deletions
+18 -6
View File
@@ -1,4 +1,4 @@
"""Tool registry: the five groups, registered in one call.
"""Tool registry: the groups, registered in one call.
The groups mirror the domain rather than the HTTP routes — papers and
paragraphs and sentences are the same tables the REST API serves, but a model
@@ -7,9 +7,10 @@ and the split between :mod:`~app.mcp.tools.papers` and
:mod:`~app.mcp.tools.paragraphs` follows those jobs.
Prefixes are load-bearing: every tool name starts with its group
(``paper_``, ``paragraph_``, ``sentence_``, ``template_``, ``field_``), because
that prefix is what a model uses to pick a tool out of a list of twenty-nine
without reading all twenty-nine descriptions.
(``paper_``, ``paragraph_``, ``sentence_``, ``template_``, ``field_``,
``reference_``, ``reference_tag_``, ``image_``), because that prefix is what a
model uses to pick a tool out of a list of forty without reading all forty
descriptions.
"""
from __future__ import annotations
@@ -18,6 +19,7 @@ from mcp.server.mcpserver import MCPServer
from app.mcp.tools import (
fields,
images,
papers,
paragraphs,
reference_tags,
@@ -28,8 +30,18 @@ from app.mcp.tools import (
#: Registration order is also the order a client lists the tools in, which is
#: the order a paper is actually written in: find it, shape it, write it, then
#: the things it is written against.
MODULES = (papers, paragraphs, sentences, templates, fields, references, reference_tags)
#: the things it is written against — and last the figure, which is written by
#: putting a sentence somewhere.
MODULES = (
papers,
paragraphs,
sentences,
templates,
fields,
references,
reference_tags,
images,
)
def register_all(server: MCPServer) -> None:
+191
View File
@@ -0,0 +1,191 @@
"""Image tools (插图): put a figure in the bucket, and hand back the sentence.
A picture is not text, so it does not live in the database: the sentence keeps a
Markdown address and the bytes live in MinIO (see :mod:`app.core.storage`). That
is what keeps a sentence one editable string, and it is also why there is no
``image_create`` that attaches a picture to a paragraph — writing a figure *is*
writing a sentence whose content happens to be ``![图注](/api/images/…)``.
One tool, and one field of a sentence, is therefore the whole feature from a
model's side. ``image_upload`` exists for the half a model cannot do itself: it
has bytes (a screenshot the user pasted, a figure it generated) and no way to
send a multipart request. Everything after that is ordinary text.
The base64 argument is a deliberate cost. A tool argument is context tokens, and
base64 is a third larger than the bytes it carries — but the alternative is a
second ingestion path (an HTTP fetch of a URL the caller names), and a service
inside the cluster that will fetch any address it is handed is an SSRF primitive
aimed at everything beside it. A model with a picture and no base64 can post it
to ``POST /api/images`` and use the Markdown that comes back; the two produce the
same sentence, byte for byte, because both go through
:func:`app.core.storage.markdown_for`.
"""
from __future__ import annotations
import base64
import binascii
import re
from typing import Annotated
from mcp.server.mcpserver import MCPServer
from pydantic import Field
from app.core.config import get_settings
from app.core.storage import get_store, image_url, markdown_for
from app.mcp import support as s
from app.mcp.tools.registry import mcp_tool
def register(server: MCPServer) -> None:
"""Register every image tool on ``server``."""
@mcp_tool(server, title="上传插图")
def image_upload(
data_base64: Annotated[
str,
Field(
description=(
"图片内容的 base64。可以直接给 data:image/png;base64,… 这种完整形式。"
"PNG / JPEG / GIF / WebP,最大 20 MB"
)
),
],
caption: Annotated[
str | None,
Field(description="图注;不给就用 filename 的主干,都没有则留空"),
] = None,
filename: Annotated[
str | None,
Field(description="文件名,只用于取图注初值;存储的键由内容决定,与它无关"),
] = None,
width: Annotated[
str | None,
Field(
description=(
"可选宽度:70%(占栏宽)或 400px。不传则按原尺寸,最大不超过栏宽"
)
),
] = None,
) -> str:
"""上传一张图片,返回可直接写进句子的 Markdown 与那个句子对象。"""
data = _decode(data_base64)
settings = get_settings()
if len(data) > settings.max_image_bytes:
s.fail(
f"图片太大了({len(data) / (1024 * 1024):.1f} MB),"
f"上限是 {settings.max_image_bytes // (1024 * 1024)} MB。先压缩或裁剪一下再传。"
)
# The store sniffs the format out of the bytes and refuses anything that
# is not a picture: an SVG is stored XSS when it is served from this
# origin, and the caller's content type or filename is only a claim.
key, content_type, digest = get_store().save(data)
markdown = _embed(key, caption=caption or _stem(filename), width=width)
return s.dumps(
{
"key": key,
"url": image_url(key),
"markdown": markdown,
"content_type": content_type,
"size": len(data),
"sha256": digest,
# The sentence, not just the Markdown: this is what the editor
# produces when a picture is dropped into it — the figure on a
# line of its own — so a model writes a figure by putting this
# object into `sentences` and changing nothing about it.
"sentence": {
"content": markdown,
"line_break_before": True,
"formula_align": "center",
},
}
)
# --- helpers -----------------------------------------------------------------
#: How a size may be written, and what it may be. Both halves mirror
#: ``WIDTH_PATTERN`` and ``parseImageWidth`` in ``frontend/src/utils/rich.ts``,
#: which is what actually reads the attribute when the paper is printed: the
#: syntax is ``{width=70%}`` or ``{width=400px}``, digits with at most one unit,
#: pixels up to :data:`MAX_PIXELS` and percentages up to 100.
#:
#: Written twice because one side is TypeScript and one is Python, and checked
#: here rather than passed through, because the reader's answer to a value it
#: cannot parse is to print the picture at its natural size — a size that
#: silently does not apply is worse than a refusal that says so.
_WIDTH_ATTRIBUTE = re.compile(r"^\s*(?:w|width)\s*=\s*(\d{1,4})\s*(%|px)?\s*$", re.IGNORECASE)
_WIDTH_BARE = re.compile(r"^\s*(\d{1,4})\s*(%|px)?\s*$")
_MAX_PIXELS = 4000
_MAX_PERCENT = 100
def _decode(value: str) -> bytes:
"""The bytes behind a base64 argument, refusing anything that is not one.
A ``data:`` prefix is stripped rather than refused: it is what a browser
writes when a picture is copied as a data URL, and it is what a model
produces when it repeats what it was given.
"""
text = value.strip()
if text.startswith("data:"):
_, _, text = text.partition(",")
# Whitespace is folded out because base64 arrives wrapped in as many ways as
# there are tools that produced it.
text = "".join(text.split())
if not text:
s.fail("data_base64 是空的,没有图片内容")
try:
return base64.b64decode(text, validate=True)
except (binascii.Error, ValueError):
s.fail("data_base64 不是合法的 base64;请给图片字节的 base64,不要给文件路径或 URL")
def _stem(filename: str | None) -> str:
"""The caption a filename suggests, the way the upload endpoint reads it."""
if not filename:
return ""
name = filename.replace("\\", "/").rsplit("/", 1)[-1]
return name.rsplit(".", 1)[0] if "." in name else name
def _embed(key: str, *, caption: str, width: str | None) -> str:
"""The Markdown a sentence carries: the address, plus an optional width.
``markdown_for`` writes the ``![图注](地址)`` half — the same function the
upload endpoint answers with — and this adds the size, which the endpoint
does not take because a browser appends it in the editor.
"""
markdown = markdown_for(key, caption)
attribute = _width_attribute(width)
return f"{markdown}{attribute}" if attribute else markdown
def _width_attribute(width: str | None) -> str:
"""``{width=70%}`` from what the caller wrote, or a refusal saying why not.
Tolerant about how it is written — ``70%``, ``70 %``, ``{width=70%}`` and
``w=400px`` all mean the same thing, which is what a model repeats back from
a sentence it read — and strict about what it means.
"""
if width is None or not str(width).strip():
return ""
written = str(width).strip().strip("{}").strip()
match = _WIDTH_ATTRIBUTE.match(written) or _WIDTH_BARE.match(written)
if match is None:
s.fail(
f"width 只能是百分比或像素,例如 70% 或 400px,收到的是“{width}”。"
"不写则按原尺寸显示(最多不超过栏宽)"
)
digits, unit = int(match.group(1)), (match.group(2) or "px").lower()
limit = _MAX_PERCENT if unit == "%" else _MAX_PIXELS
if not 1 <= digits <= limit:
s.fail(f"width 的 {unit} 值要在 1 到 {limit} 之间,收到的是“{width}”")
return f"{{width={digits}{unit}}}"
__all__ = ["register"]
+4 -4
View File
@@ -342,12 +342,12 @@ def register(server: MCPServer) -> None:
paper_crud.append_sentence(
db,
paper,
# Spread, not copied field by field: a sentence
# column added to the domain then reaches this path
# by itself, instead of being dropped in silence.
SentenceCreate(
**item.model_dump(),
paper_template_filed_sort=position,
content=item.content,
line_break_before=item.line_break_before,
citations=item.citations,
sort=item.sort,
),
)
else:
+12 -17
View File
@@ -204,19 +204,13 @@ def register(server: MCPServer) -> None:
if not rows:
s.fail(f"第 {source} 段没有内容可以搬")
# Moved, not rewritten. Every column of the sentence travels —
# where a line started, how a formula was aligned, what it cited —
# because all of it is part of what was there; and ``sort`` is the
# one field deliberately left behind (see
# :func:`app.mcp.support.sentence_values`).
inputs = [
SentenceInput(
content=row.content,
# ``sort`` is deliberately left unset so that
# :func:`app.crud.paper.replace_paragraph` numbers the block
# from the *target's* current maximum. Carrying the old
# numbers over instead would interleave the two paragraphs —
# the moved sentence with ``sort=1`` would sort against the
# target's own sentence with ``sort=1`` — and the merge would
# read in an order neither writer wrote.
line_break_before=row.line_break_before,
citations=list(row.citations),
)
SentenceInput(**s.sentence_values(row), citations=list(row.citations))
for row in rows
]
paper_crud.replace_paragraph(
@@ -266,6 +260,11 @@ def _write(
at a time — the same call ``POST /papers/{id}/sentences`` makes — rather
than read-modify-write, so two sentences appended in one request cannot
overwrite each other's ``sort``.
The payload is spread rather than copied field by field, so a column added
to a sentence reaches this path without anyone remembering to add it here —
which is how ``line_break_before`` and ``formula_align`` each went missing
once.
"""
if mode == "append":
for item in inputs:
@@ -273,11 +272,7 @@ def _write(
db,
paper,
SentenceCreate(
paper_template_filed_sort=field_sort,
content=item.content,
line_break_before=item.line_break_before,
citations=item.citations,
sort=item.sort,
**item.model_dump(), paper_template_filed_sort=field_sort
),
)
else:
+14 -1
View File
@@ -32,6 +32,7 @@ from typing import Any, TypeVar
from mcp.server.mcpserver import MCPServer
from mcp.server.mcpserver.exceptions import ToolError
from app.core.storage import StorageUnavailable, UnsupportedImage
from app.crud.reference import UnknownReference
from app.crud.reference_tag import DuplicateTagName, UnknownTag
@@ -46,7 +47,19 @@ F = TypeVar("F", bound=Callable[..., Any])
#: would have let 「标签不存在:9」 reach the model as an unreadable
#: "Error executing tool ...", which is the exact failure this wrapper exists to
#: prevent. A new domain refusal belongs in this tuple on the day it is written.
DOMAIN_REFUSALS = (UnknownReference, UnknownTag, DuplicateTagName)
#:
#: The two storage ones are here for the same reason. A bucket that is not
#: configured and a file that is not a picture are both answers rather than
#: faults — "缺少 MINIO_ENDPOINT" and "只接受 PNG / JPEG / GIF / WebP" are the
#: two things a caller can act on, and neither survives the wrapper as a stack
#: trace.
DOMAIN_REFUSALS = (
UnknownReference,
UnknownTag,
DuplicateTagName,
StorageUnavailable,
UnsupportedImage,
)
def mcp_tool(
+13 -2
View File
@@ -20,7 +20,7 @@ from pydantic import Field
from app.crud import paper as paper_crud
from app.mcp import support as s
from app.mcp.specs import CitationSpec
from app.mcp.specs import CitationSpec, FormulaAlign
from app.mcp.tools.registry import mcp_tool
from app.models import Paper
from app.schemas.paper import CitationInput, SentenceCreate, SentenceUpdate
@@ -94,6 +94,10 @@ def register(server: MCPServer) -> None:
)
),
] = False,
formula_align: Annotated[
FormulaAlign,
Field(description="display 公式怎么排:center(默认)/ left / right"),
] = "center",
sort: Annotated[
int | None, Field(description="段内位置;不传则排在段尾")
] = None,
@@ -109,6 +113,7 @@ def register(server: MCPServer) -> None:
paper_template_filed_sort=field_sort,
content=content,
line_break_before=line_break_before,
formula_align=formula_align,
citations=_citations(citations or []),
sort=sort,
),
@@ -137,17 +142,22 @@ def register(server: MCPServer) -> None:
bool | None,
Field(description="这一句是否另起一行;不传则不动,false 表示取消换行"),
] = None,
formula_align: Annotated[
FormulaAlign | None,
Field(description="display 公式怎么排;不传则不动"),
] = None,
citations: Annotated[
list[CitationSpec] | None,
Field(description="引用整组替换;传 [] 表示清空引用"),
] = None,
) -> str:
"""修改一句话的正文、所属段落、排序、换行或引用(引用是整体替换)。"""
"""修改一句话的正文、所属段落、排序、换行、公式对齐或引用(引用是整体替换)。"""
if (
content is None
and position is None
and sort is None
and line_break_before is None
and formula_align is None
and citations is None
):
s.fail("没有给出任何要修改的内容")
@@ -164,6 +174,7 @@ def register(server: MCPServer) -> None:
paper_template_filed_sort=position,
sort=sort,
line_break_before=line_break_before,
formula_align=formula_align,
citations=None if citations is None else _citations(citations),
),
)