48f0c12fd2
Figures come out of a plotting tool at whatever size they were made. On the
paper this was written for, the sixteen of them are 1441, 1857, 2409 and 4169
pixels wide — every one wider than the 1276px column — so with no size they all
printed at the full width, and the near-square ones (1857×1986, 1974×2068) took
1365px of height, a page each. Measured in the browser, all sixteen came out at
exactly 1276px: not one of them was the size its author wanted.
`{width=…}` has been in the syntax all along, so this is the control that was
missing, not the feature. A row that is a figure now carries 宽度 in its header:
25/50/70/100%, or any value typed in — the size a panel wants is what it
measures, not a round number — with 原尺寸 as the empty value. Beside it, what
the picture will measure on the page (`原图 1857×1986 · 显示 893×955`), worked
out against the column actually rendered behind the dialog, which is the number
the writer is deciding with.
Two rules keep it honest. The value goes through `parseImageWidth` — the same
function that parses the attribute when the paper is printed — so what the
control shows is what will be printed, and a value it cannot read is refused
with the forms that work instead of stored as a size that quietly does nothing.
And the attribute is written into the sentence content by `withImageWidth`,
which edits only the first picture: the control is offered for a figure, and a
sentence with words around its picture is not one. Reading the size back out of
the content rather than keeping a copy of it in the draft means there is one
place it can be wrong instead of two.
`check-image.mjs` grows from 15 checks to 24, driving the real control: that
only figure rows offer one, that it shows what each is set to, that typing 30%
converts to 383×239 immediately, that what was typed is what gets stored, that
the paper then prints it at 30% of the column, that clearing it goes back to the
natural 480×300, and that "宽一点" is refused while the sentence keeps its size.
The MCP side of the same rule was wrong and is fixed with it: `image_upload(width=
"60")` wrote `{width=60px}`, and the reader takes a bare number as a share of the
column — `parseImageWidth` only reads pixels when the value says `px`. A model
asking for 60% got a 60-pixel figure. v7 is deployed with the corrected reading,
and smoke_mcp.py checks it.
205 lines
8.6 KiB
Python
205 lines
8.6 KiB
Python
"""Image tools (插图): put a figure in the bucket, and hand back the sentence.
|
|
|
|
A picture is not text, so it does not live in the database: the sentence keeps a
|
|
Markdown address and the bytes live in MinIO (see :mod:`app.core.storage`). That
|
|
is what keeps a sentence one editable string, and it is also why there is no
|
|
``image_create`` that attaches a picture to a paragraph — writing a figure *is*
|
|
writing a sentence whose content happens to be ````.
|
|
|
|
One tool, and one field of a sentence, is therefore the whole feature from a
|
|
model's side. ``image_upload`` exists for the half a model cannot do itself: it
|
|
has bytes (a screenshot the user pasted, a figure it generated) and no way to
|
|
send a multipart request. Everything after that is ordinary text.
|
|
|
|
The base64 argument is a deliberate cost. A tool argument is context tokens, and
|
|
base64 is a third larger than the bytes it carries — but the alternative is a
|
|
second ingestion path (an HTTP fetch of a URL the caller names), and a service
|
|
inside the cluster that will fetch any address it is handed is an SSRF primitive
|
|
aimed at everything beside it. A model with a picture and no base64 can post it
|
|
to ``POST /api/images`` and use the Markdown that comes back; the two produce the
|
|
same sentence, byte for byte, because both go through
|
|
:func:`app.core.storage.markdown_for`.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import base64
|
|
import binascii
|
|
import re
|
|
from typing import Annotated
|
|
|
|
from mcp.server.mcpserver import MCPServer
|
|
from pydantic import Field
|
|
|
|
from app.core.config import get_settings
|
|
from app.core.storage import get_store, image_url, markdown_for
|
|
from app.mcp import support as s
|
|
from app.mcp.tools.registry import mcp_tool
|
|
|
|
|
|
def register(server: MCPServer) -> None:
|
|
"""Register every image tool on ``server``."""
|
|
|
|
@mcp_tool(server, title="上传插图")
|
|
def image_upload(
|
|
data_base64: Annotated[
|
|
str,
|
|
Field(
|
|
description=(
|
|
"图片内容的 base64。可以直接给 data:image/png;base64,… 这种完整形式。"
|
|
"PNG / JPEG / GIF / WebP,最大 20 MB"
|
|
)
|
|
),
|
|
],
|
|
caption: Annotated[
|
|
str | None,
|
|
Field(description="图注;不给就用 filename 的主干,都没有则留空"),
|
|
] = None,
|
|
filename: Annotated[
|
|
str | None,
|
|
Field(description="文件名,只用于取图注初值;存储的键由内容决定,与它无关"),
|
|
] = None,
|
|
width: Annotated[
|
|
str | None,
|
|
Field(
|
|
description=(
|
|
"可选宽度:70%(占栏宽)或 400px;只写数字按百分比算。"
|
|
"不传则按原尺寸,最大不超过栏宽"
|
|
)
|
|
),
|
|
] = None,
|
|
) -> str:
|
|
"""上传一张图片,返回可直接写进句子的 Markdown 与那个句子对象。"""
|
|
data = _decode(data_base64)
|
|
settings = get_settings()
|
|
if len(data) > settings.max_image_bytes:
|
|
s.fail(
|
|
f"图片太大了({len(data) / (1024 * 1024):.1f} MB),"
|
|
f"上限是 {settings.max_image_bytes // (1024 * 1024)} MB。先压缩或裁剪一下再传。"
|
|
)
|
|
|
|
# The store sniffs the format out of the bytes and refuses anything that
|
|
# is not a picture: an SVG is stored XSS when it is served from this
|
|
# origin, and the caller's content type or filename is only a claim.
|
|
key, content_type, digest = get_store().save(data)
|
|
|
|
markdown = _embed(key, caption=caption or _stem(filename), width=width)
|
|
return s.dumps(
|
|
{
|
|
"key": key,
|
|
"url": image_url(key),
|
|
"markdown": markdown,
|
|
"content_type": content_type,
|
|
"size": len(data),
|
|
"sha256": digest,
|
|
# The sentence, not just the Markdown: this is what the editor
|
|
# produces when a picture is dropped into it — the figure on a
|
|
# line of its own — so a model writes a figure by putting this
|
|
# object into `sentences` and changing nothing about it.
|
|
"sentence": {
|
|
"content": markdown,
|
|
"line_break_before": True,
|
|
"formula_align": "center",
|
|
},
|
|
}
|
|
)
|
|
|
|
|
|
# --- helpers -----------------------------------------------------------------
|
|
|
|
#: How a size may be written, and what it may be. Both halves mirror
|
|
#: ``WIDTH_PATTERN`` and ``parseImageWidth`` in ``frontend/src/utils/rich.ts``,
|
|
#: which is what actually reads the attribute when the paper is printed: the
|
|
#: syntax is ``{width=70%}`` or ``{width=400px}``, digits with at most one unit,
|
|
#: pixels up to :data:`MAX_PIXELS` and percentages up to 100.
|
|
#:
|
|
#: Written twice because one side is TypeScript and one is Python, and checked
|
|
#: here rather than passed through, because the reader's answer to a value it
|
|
#: cannot parse is to print the picture at its natural size — a size that
|
|
#: silently does not apply is worse than a refusal that says so.
|
|
_WIDTH_ATTRIBUTE = re.compile(r"^\s*(?:w|width)\s*=\s*(\d{1,4})\s*(%|px)?\s*$", re.IGNORECASE)
|
|
_WIDTH_BARE = re.compile(r"^\s*(\d{1,4})\s*(%|px)?\s*$")
|
|
_MAX_PIXELS = 4000
|
|
_MAX_PERCENT = 100
|
|
|
|
|
|
def _decode(value: str) -> bytes:
|
|
"""The bytes behind a base64 argument, refusing anything that is not one.
|
|
|
|
A ``data:`` prefix is stripped rather than refused: it is what a browser
|
|
writes when a picture is copied as a data URL, and it is what a model
|
|
produces when it repeats what it was given.
|
|
"""
|
|
text = value.strip()
|
|
if text.startswith("data:"):
|
|
_, _, text = text.partition(",")
|
|
# Whitespace is folded out because base64 arrives wrapped in as many ways as
|
|
# there are tools that produced it.
|
|
text = "".join(text.split())
|
|
if not text:
|
|
s.fail("data_base64 是空的,没有图片内容")
|
|
try:
|
|
return base64.b64decode(text, validate=True)
|
|
except (binascii.Error, ValueError):
|
|
s.fail("data_base64 不是合法的 base64;请给图片字节的 base64,不要给文件路径或 URL")
|
|
|
|
|
|
def _stem(filename: str | None) -> str:
|
|
"""The caption a filename suggests, the way the upload endpoint reads it."""
|
|
if not filename:
|
|
return ""
|
|
name = filename.replace("\\", "/").rsplit("/", 1)[-1]
|
|
return name.rsplit(".", 1)[0] if "." in name else name
|
|
|
|
|
|
def _embed(key: str, *, caption: str, width: str | None) -> str:
|
|
"""The Markdown a sentence carries: the address, plus an optional width.
|
|
|
|
``markdown_for`` writes the ```` half — the same function the
|
|
upload endpoint answers with — and this adds the size, which the endpoint
|
|
does not take because a browser appends it in the editor.
|
|
"""
|
|
markdown = markdown_for(key, caption)
|
|
attribute = _width_attribute(width)
|
|
return f"{markdown}{attribute}" if attribute else markdown
|
|
|
|
|
|
def _width_attribute(width: str | None) -> str:
|
|
"""``{width=70%}`` from what the caller wrote, or a refusal saying why not.
|
|
|
|
Tolerant about how it is written — ``70%``, ``70 %``, ``{width=70%}`` and
|
|
``w=400px`` all mean the same thing, which is what a model repeats back from
|
|
a sentence it read — and strict about what it means.
|
|
|
|
A bare number is a **percentage**, because that is what the reader makes of
|
|
one: ``parseImageWidth`` only takes a value as pixels when it is written
|
|
``px``, and treats everything else as a share of the column. ``400`` is
|
|
therefore out of range and refused with the two forms that work, rather than
|
|
written as ``400px`` and printed at a third of the page.
|
|
"""
|
|
if width is None or not str(width).strip():
|
|
return ""
|
|
|
|
written = str(width).strip().strip("{}").strip()
|
|
match = _WIDTH_ATTRIBUTE.match(written) or _WIDTH_BARE.match(written)
|
|
if match is None:
|
|
s.fail(
|
|
f"width 只能是百分比或像素,例如 70% 或 400px,收到的是“{width}”。"
|
|
"不写则按原尺寸显示(最多不超过栏宽)"
|
|
)
|
|
|
|
digits = int(match.group(1))
|
|
unit = (match.group(2) or "%").lower()
|
|
limit = _MAX_PERCENT if unit == "%" else _MAX_PIXELS
|
|
if not 1 <= digits <= limit:
|
|
# The pointer to pixels only helps when the caller wrote a bare number:
|
|
# "150%" is already written as a percentage and does not want advice.
|
|
hint = f"像素要写成 “{digits}px”" if match.group(2) is None else ""
|
|
s.fail(
|
|
f"width 的 {unit} 值要在 1 到 {limit} 之间,收到的是“{width}”。{hint}"
|
|
)
|
|
return f"{{width={digits}{unit}}}"
|
|
|
|
|
|
__all__ = ["register"]
|