Files
paper-doc/backend/scripts/smoke_mcp.py
T
govin ebaf9e558b backend: let a model put a figure in the paper, and set a formula's alignment
Two things the editor gained and MCP could not do, both because the tool layer
had been told about the domain's columns by hand.

**A figure.** Bytes go to MinIO and the sentence keeps a Markdown address, so a
picture is ordinary text — which means the writing tools already carry it, and
the only half a model cannot do is the upload: it has bytes and no way to send
multipart. `image_upload` takes base64 and answers with the URL, the Markdown
and the *sentence object* the editor produces when a picture is dropped in, so a
figure is written by putting that object into `sentences` unchanged. It does not
fetch a URL the caller names: a service inside the cluster that will GET any
address it is handed is an SSRF primitive aimed at everything beside it.

The width argument is the reader's rule or it is refused. `{width=70%}` is
parsed by `frontend/src/utils/rich.ts`, and a value it cannot read is not an
error there — the picture just prints at its natural size. A size that silently
does not apply is worse than a refusal, so the syntax and the range (pixels
1–4000, per cent 1–100) are mirrored in `app/mcp/tools/images.py`, with the
usual comment naming the other side.

`paragraph_text` also had to learn what a figure is: a sentence holding nothing
but an image is a 插图 — the view prints it on its own line with the brackets'
text as its caption — and the text a tool hands back broke lines around a
display formula but not around a picture. Same rule as `figureOf`, same pair of
implementations, same warning about changing both.

**A formula's alignment.** `paper_sentence.formula_align` reached the editor and
the REST API and stopped there: `SentenceSpec` had no such field, so a model that
sent one got a successful call and a centred formula. The column is now declared,
carried through every write path and reported by every read path.

That is the second column lost the same way, so the fix is structural rather
than another two edits. `SENTENCE_FIELDS` is read from
`app.schemas.paper.SentenceInput` instead of written out, the payload is built
by one `sentence_payload()`, and the three append paths spread
`SentenceCreate(**item.model_dump())` rather than copying fields. One edit
remains — declaring the field in `specs.py` — and smoke_mcp.py's first step
compares the two field sets, so forgetting it now fails the suite instead of
losing the field. `sort` is declared too, which it never was: a model that sent
one had it dropped in silence.

smoke_mcp.py: 99 checks to 119. The figure step uploads a real 1x1 GIF (the same
bytes twice, and the key is the same object), writes the returned sentence as it
came back, checks the paragraph prints it alone on its line and that an image
inside a sentence does not break the line, and takes the four refusals — not a
picture, an unparseable size, an out-of-range one, not base64. The alignment step
writes a right-aligned formula, moves the sentence to left on its own, and takes
the paragraph to another position to prove the setting travels with it.

Deployed as v6, and the pod now carries MINIO_ENDPOINT (the in-cluster Service)
plus the two credential keys from the Secret; without them the image tool answers
with the missing variable names and every writing tool still works.
2026-09-24 10:54:27 +08:00

1340 lines
54 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""End-to-end smoke test for the MCP server, spoken as a real MCP client.
Run it from ``backend/``::
.venv/bin/python scripts/smoke_mcp.py
It spawns ``scripts/mcp_server.py`` exactly the way Claude Code, Codex and the
Harness do — a child process, JSON-RPC over its stdin/stdout — and then walks
the whole writing loop through tool calls: create a paper on a template, read
the outline, write paragraphs by *heading*, write a whole generated document in
one call, edit one sentence without rewriting its paragraph, search across
papers, switch the template, and delete everything it created.
Speaking the real protocol is the point. Calling the tool functions directly
would test the domain layer, which ``scripts/smoke_papers.py`` already covers;
what is untested until a client connects is the part that only exists on the
wire — argument schemas, tool discovery, error results, and the stdio stream
staying clean enough to carry the protocol.
Exits non-zero on the first failed expectation and cleans up after itself.
"""
from __future__ import annotations
import argparse
import base64
import json
import os
import re
import sys
from collections.abc import AsyncIterator, Iterator
from contextlib import asynccontextmanager, contextmanager
from pathlib import Path
from typing import Any
import anyio
BACKEND_DIR = Path(__file__).resolve().parents[1]
if str(BACKEND_DIR) not in sys.path:
sys.path.insert(0, str(BACKEND_DIR))
from mcp import types # noqa: E402
from mcp.client.session import ClientSession # noqa: E402
from mcp.client.stdio import StdioServerParameters, stdio_client # noqa: E402
from sqlalchemy import or_, select # noqa: E402
from app.db.session import SessionLocal # noqa: E402
from app.mcp.specs import SentenceSpec # noqa: E402
from app.models import Paper, Reference, ReferenceTag, Template # noqa: E402
from app.schemas.paper import SentenceInput # noqa: E402
SERVER_SCRIPT = BACKEND_DIR / "scripts" / "mcp_server.py"
#: A 1x1 GIF, and something that is definitely not a picture. Both are inline
#: rather than files on disk: the suite has to run from a checkout with nothing
#: beside it, and a fixture that cannot go missing cannot fail for the wrong
#: reason.
GIF_1PX_BASE64 = "R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7"
NOT_A_PICTURE_BASE64 = base64.b64encode(
b'<svg xmlns="http://www.w3.org/2000/svg"><script>alert(1)</script></svg>'
).decode()
_checks = 0
_created_papers: list[int] = []
_created_templates: list[int] = []
_created_references: list[int] = []
_created_tags: list[int] = []
def check(label: str, condition: bool, detail: str = "") -> None:
"""Assert one expectation and report it."""
global _checks
_checks += 1
if not condition:
raise AssertionError(f"{label}: {detail}")
print(f" ok {label}")
def payload(result: types.CallToolResult) -> Any:
"""The JSON a tool returned, refusing an error result.
A tool that failed returns ``isError`` with the message as its content, not
an exception — so a smoke test that only looked at the content would read a
refusal as data. That is the failure mode this helper exists to prevent.
"""
text = "".join(
block.text for block in result.content if isinstance(block, types.TextContent)
)
if result.is_error:
raise AssertionError(f"tool returned an error: {text}")
return json.loads(text) if text else None
async def call(session: ClientSession, tool: str, **arguments: Any) -> Any:
"""Call one tool and return its JSON payload.
The parameter is ``tool``, not ``name``: several tools here take an
argument called ``name`` (``template_create``, ``field_create``) and a
helper whose own parameter shadowed it would fail with a TypeError that
looks nothing like the tool call it came from.
"""
return payload(await session.call_tool(tool, arguments))
async def expect_error(session: ClientSession, tool: str, **arguments: Any) -> str:
"""Call one tool expecting a refusal, and return the message."""
result = await session.call_tool(tool, arguments)
if not result.is_error:
raise AssertionError(f"{tool} should have refused, but returned {result.content}")
return "".join(
block.text for block in result.content if isinstance(block, types.TextContent)
)
def pick_template() -> tuple[int, list[dict[str, Any]]]:
"""The template with the most titles, and its outline as the tools see it."""
with SessionLocal() as db:
template = db.scalars(
select(Template).order_by(Template.id.asc())
).all()
best = max(template, key=lambda item: len(item.placements))
return best.id, [
{"position": item.sort, "name": item.title.name}
for item in best.placements
]
@asynccontextmanager
async def connect(url: str | None, token: str | None) -> AsyncIterator[ClientSession]:
"""Open a session over stdio (default) or against a running HTTP server.
Both paths yield the same :class:`ClientSession`, so every check below runs
unchanged on either transport — which is the point of testing the protocol
rather than the functions: the tools must behave identically whether a
client spawned them or connected to them.
"""
if url:
# headers live on the HTTP client, not on the transport call.
import httpx2
from mcp.client.streamable_http import streamable_http_client
headers = {"Authorization": f"Bearer {token}"} if token else {}
async with httpx2.AsyncClient(headers=headers) as http_client:
async with streamable_http_client(url, http_client=http_client) as (
read,
write,
):
async with ClientSession(read, write) as session:
yield session
return
parameters = StdioServerParameters(
command=sys.executable,
args=[str(SERVER_SCRIPT)],
cwd=str(BACKEND_DIR),
)
async with stdio_client(parameters) as (read, write):
async with ClientSession(read, write) as session:
yield session
async def run(url: str | None = None, token: str | None = None) -> int:
return await _run(url, token)
async def _run(url: str | None = None, token: str | None = None) -> int:
template_id, outline = pick_template()
# Four headings, because the line-break and bibliography steps below move one
# paragraph onto a fourth; with fewer, they would have nowhere to write.
check("a template with four fields exists", len(outline) >= 4, f"got {len(outline)}")
first = outline[0]["name"]
second = outline[1]["name"]
async with connect(url, token) as session:
handshake = await session.initialize()
check(
"server announces itself",
handshake.server_info.name == "paper-doc",
str(handshake.server_info),
)
print("\n1. tool discovery")
listed = await session.list_tools()
names = {tool.name for tool in listed.tools}
check("the whole surface is published", len(names) == 42, f"got {len(names)}")
check(
"every group is reachable",
{
"paper_write",
"paragraph_write",
"sentence_add",
"template_get",
"field_list",
"reference_create",
"reference_tag_create",
"image_upload",
}
<= names,
", ".join(sorted(names)),
)
# The prefix is how a model finds a tool in a list of forty: a tool whose
# name does not start with its group's prefix is a tool nothing reaches
# for, and the mistake is invisible in every other check.
check(
"every tool name starts with its group's prefix",
all(
name.startswith(
(
"paper_",
"paragraph_",
"sentence_",
"template_",
"field_",
"reference_",
"reference_tag_",
"image_",
)
)
for name in names
),
", ".join(sorted(names)),
)
check(
"no tool returns a structured envelope",
all(tool.output_schema is None for tool in listed.tools),
)
write_tool = next(tool for tool in listed.tools if tool.name == "paper_write")
check(
"paper_write describes its paragraph array",
"paragraphs" in write_tool.input_schema["properties"],
)
# A sentence column that this surface does not declare is a column no
# model can set — which is exactly how `line_break_before` and then
# `formula_align` were lost: the write succeeded and the field was
# silently gone. Declaring every field is one line; forgetting one fails
# here.
missing = set(SentenceInput.model_fields) - set(SentenceSpec.model_fields)
check(
"the sentence schema declares every writable field of a sentence",
not missing,
f"missing from SentenceSpec: {sorted(missing)}",
)
add_tool = next(tool for tool in listed.tools if tool.name == "sentence_add")
align = add_tool.input_schema["properties"]["formula_align"]
check(
"and a formula's alignment is an enum, so a model cannot invent one",
align.get("enum") == ["center", "left", "right"],
json.dumps(align, ensure_ascii=False),
)
print("\n2. create a paper on that template")
paper = await call(
session,
"paper_create",
title="SMOKE MCP 论文",
template_id=template_id,
author="smoke",
keywords="测试, smoke, 测试",
)
_created_papers.append(paper["id"])
check("created", paper["title"] == "SMOKE MCP 论文")
check("keywords are deduplicated", paper["keywords"] == "测试, smoke", paper["keywords"])
check(
"the whole structure exists with no content",
paper["paragraphs"] == 0 and paper["template_paragraphs"] == len(outline),
json.dumps(paper, ensure_ascii=False),
)
print("\n3. read the outline before writing")
skeleton = await call(session, "paper_outline", paper_id=paper["id"])
check(
"the outline carries every position",
len(skeleton["paragraphs"]) == len(outline),
json.dumps(skeleton, ensure_ascii=False),
)
check("nothing has been written", all(item["sentences"] == 0 for item in skeleton["paragraphs"]))
print("\n4. write paragraphs by heading")
written = await call(
session,
"paper_write",
paper_id=paper["id"],
paragraphs=[
{
"heading": first,
"sentences": ["第一句。", "第二句。"],
},
{
"heading": second.lstrip("0123456789. "),
"text": "这一整段只有一行,按句号切成两句。第二句在这里。",
"split": "sentence",
},
],
)
check("two paragraphs written", len(written["written"]) == 2, json.dumps(written, ensure_ascii=False))
check("four sentences total", written["total_sentences"] == 4, str(written["total_sentences"]))
check(
"both headings resolved to template positions",
all(item["heading"] for item in written["written"]),
json.dumps(written, ensure_ascii=False),
)
print("\n5. every read tool answers")
# The read-only tools are what a suite forgets, and they are where a
# rename bites: `template_list` read a schema attribute that no longer
# existed and crashed with an unreadable "Error executing tool" while
# every other check here passed, because nothing called it. Each
# assertion below is about the *shape* a caller depends on, not merely
# that the call came back.
templates = await call(session, "template_list", page_size=50)
check(
"template_list lists the paper's template",
any(item["id"] == template_id for item in templates["items"]),
json.dumps(templates, ensure_ascii=False)[:300],
)
check(
"and reports how many fields it places",
all(isinstance(item.get("fields"), int) for item in templates["items"]),
json.dumps(templates["items"], ensure_ascii=False)[:300],
)
fetched_template = await call(session, "template_get", template_id=template_id)
positions = [item["position"] for item in fetched_template["fields"]]
check(
"template_get returns the outline already ordered",
positions == sorted(positions) and len(positions) == len(outline),
json.dumps(positions),
)
library = await call(session, "field_list", page_size=5)
check(
"field_list pages the library",
library["total"] >= 1 and len(library["items"]) <= 5,
json.dumps({key: library[key] for key in ("total", "page_size", "pages")}),
)
one_field = await call(session, "field_get", field_id=library["items"][0]["id"])
check(
"field_get returns that same entry",
one_field["id"] == library["items"][0]["id"],
json.dumps(one_field, ensure_ascii=False),
)
listed_papers = await call(session, "paper_list", keyword="SMOKE MCP")
check(
"paper_list finds the paper by title",
any(item["id"] == paper["id"] for item in listed_papers["items"]),
json.dumps(listed_papers, ensure_ascii=False)[:300],
)
fetched_paper = await call(session, "paper_get", paper_id=paper["id"])
check(
"paper_get reports the four sentences written so far",
fetched_paper["sentences"] == 4,
json.dumps(fetched_paper, ensure_ascii=False),
)
paragraphs = await call(session, "paragraph_list", paper_id=paper["id"])
check(
"paragraph_list covers every position the template defines",
len(paragraphs["items"]) == len(positions),
f"{len(paragraphs['items'])} vs {len(positions)}",
)
print("\n6. a heading the template does not have is refused, with candidates")
message = await expect_error(
session,
"paragraph_write",
paper_id=paper["id"],
heading="不存在的标题",
sentences=["x"],
)
check("the refusal lists the real headings", first in message, message)
print("\n7. read one paragraph back")
paragraph = await call(
session, "paragraph_get", paper_id=paper["id"], heading=first
)
check("two sentences stored", len(paragraph["sentences"]) == 2, json.dumps(paragraph, ensure_ascii=False))
check(
"CJK seams take no separator",
all(row["separator_before"] == "" for row in paragraph["sentences"]),
json.dumps(paragraph["sentences"], ensure_ascii=False),
)
print("\n8. the reference library")
tag = await call(
session, "reference_tag_create", name="SMOKE MCP 标签", color="#123456"
)
_created_tags.append(tag["id"])
check("a tag can be created", tag["name"] == "SMOKE MCP 标签", json.dumps(tag, ensure_ascii=False))
created_reference = await call(
session,
"reference_create",
reference={
"type": "journal-article",
"title": "SMOKE MCP 文献",
"container_title": "Journal of Smoke Tests",
"publication_year": 2024,
"volume": "12",
"issue": "3",
"pages": "45-67",
"doi": "10.1000/smoke.mcp.001",
"language": "en",
"authors": [
{"name": "Smith, John", "family_name": "Smith", "given_name": "John"},
{"name": "张三", "role": "editor"},
],
"tag_ids": [tag["id"]],
},
)
reference_id = created_reference["id"]
_created_references.append(reference_id)
check(
"authors keep the order they were sent in",
[author["name"] for author in created_reference["authors"]] == ["Smith, John", "张三"],
json.dumps(created_reference["authors"], ensure_ascii=False),
)
check(
"and the tag is attached",
[item["name"] for item in created_reference["tags"]] == ["SMOKE MCP 标签"],
json.dumps(created_reference["tags"], ensure_ascii=False),
)
tags = await call(session, "reference_tag_list", keyword="SMOKE MCP")
check(
"reference_tag_list counts the literature under the tag",
any(item["id"] == tag["id"] and item["references"] == 1 for item in tags["items"]),
json.dumps(tags, ensure_ascii=False)[:300],
)
one_tag = await call(session, "reference_tag_get", tag_id=tag["id"])
check(
"reference_tag_get returns that same tag",
one_tag["id"] == tag["id"] and one_tag["references"] == 1,
json.dumps(one_tag, ensure_ascii=False),
)
retagged = await call(
session,
"reference_tag_update",
tag_id=tag["id"],
color="#ABCDEF",
note="测试备注",
)
check(
"reference_tag_update changes only what it was given",
retagged["color"] == "#ABCDEF"
and retagged["note"] == "测试备注"
and retagged["name"] == "SMOKE MCP 标签",
json.dumps(retagged, ensure_ascii=False),
)
unlabelled_note = await call(
session, "reference_tag_update", tag_id=tag["id"], clear_note=True
)
check(
"clear_note takes the note back off",
"note" not in unlabelled_note,
json.dumps(unlabelled_note, ensure_ascii=False),
)
by_doi = await call(session, "reference_list", keyword="10.1000/smoke.mcp.001")
check(
"reference_list finds it by DOI",
any(item["id"] == reference_id for item in by_doi["items"]),
json.dumps(by_doi, ensure_ascii=False)[:300],
)
check(
"a row carries its citation count",
all(isinstance(item.get("citations"), int) for item in by_doi["items"]),
json.dumps(by_doi["items"], ensure_ascii=False)[:300],
)
fetched_reference = await call(session, "reference_get", reference_id=reference_id)
check(
"reference_get returns the whole record",
fetched_reference["doi"] == "10.1000/smoke.mcp.001"
and fetched_reference["container_title"] == "Journal of Smoke Tests",
json.dumps(fetched_reference, ensure_ascii=False)[:300],
)
patched = await call(
session,
"reference_update",
reference_id=reference_id,
changes={"note": "由测试写入", "publication_year": 2025},
)
check(
"reference_update changes only what it was given",
patched["publication_year"] == 2025 and patched["title"] == "SMOKE MCP 文献",
json.dumps(patched, ensure_ascii=False)[:300],
)
cleared = await call(
session, "reference_update", reference_id=reference_id, changes={"note": None}
)
check(
"an explicit null clears the field",
"note" not in cleared,
json.dumps(cleared, ensure_ascii=False)[:200],
)
imported = await call(
session,
"reference_import",
references=[
{"title": "SMOKE MCP 导入一", "publication_year": 2020},
{"title": "SMOKE MCP 导入二", "publication_year": 2021},
{"title": "SMOKE MCP 文献", "doi": "10.1000/smoke.mcp.001"},
{"title": " ", "type": "journal-article"},
],
)
imported_ids = [item["id"] for item in imported["created"]]
_created_references.extend(imported_ids)
check(
"import creates the entries that are new",
len(imported["created"]) == 2,
json.dumps(imported, ensure_ascii=False)[:400],
)
check(
"skips the one the library already has, naming it",
len(imported["skipped"]) == 1
and imported["skipped"][0]["existing_id"] == reference_id,
json.dumps(imported["skipped"], ensure_ascii=False),
)
check(
"and reports the bad entry without losing the rest",
len(imported["failed"]) == 1 and imported["failed"][0]["title"].strip() == "",
json.dumps(imported["failed"], ensure_ascii=False),
)
print("\n9. append a sentence with a citation")
added = await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=first,
content="A cited sentence.",
citations=[{"quote": "引用的原话", "reference_id": reference_id}],
)
check("appended", added["sentence"]["content"] == "A cited sentence.")
check("the citation is stored", added["sentence"]["citations"][0]["quote"] == "引用的原话")
after = await call(session, "paragraph_get", paper_id=paper["id"], heading=first)
check(
"the seam before it takes a space",
after["sentences"][-1]["separator_before"] == " ",
json.dumps(after["sentences"][-1], ensure_ascii=False),
)
print("\n10. a cited reference cannot be deleted")
citations = await call(session, "reference_citations", reference_id=reference_id)
check(
"the reference reports the sentence citing it",
citations["total"] == 1
and citations["items"][0]["paper_id"] == paper["id"]
and citations["items"][0]["quote"] == "引用的原话",
json.dumps(citations, ensure_ascii=False)[:400],
)
message = await expect_error(session, "reference_delete", reference_ids=[reference_id])
check("deleting it is refused with the count", "1 处引用" in message, message)
cited = await call(session, "sentence_list", paper_id=paper["id"])
sentence_id = next(row["id"] for row in cited["items"] if row["citations"])
await call(
session,
"sentence_update",
paper_id=paper["id"],
sentence_id=sentence_id,
citations=[],
)
check(
"clearing the citation frees the reference",
(await call(session, "reference_citations", reference_id=reference_id))["total"] == 0,
)
freed = await call(session, "reference_delete", reference_ids=reference_id)
check("now it deletes", freed["deleted"] == 1, json.dumps(freed, ensure_ascii=False))
_created_references.remove(reference_id)
batch = await call(session, "reference_delete", reference_ids=imported_ids)
check(
"a batch delete takes the imported ones too",
batch["deleted"] == len(imported_ids),
json.dumps(batch, ensure_ascii=False),
)
for imported_id in imported_ids:
_created_references.remove(imported_id)
dropped = await call(session, "reference_tag_delete", tag_ids=tag["id"])
check("and the tag goes with it", dropped["deleted"] == 1, json.dumps(dropped, ensure_ascii=False))
_created_tags.remove(tag["id"])
print("\n11. edit one sentence without touching its paragraph")
edited = await call(
session,
"sentence_update",
paper_id=paper["id"],
sentence_id=after["sentences"][0]["id"],
content="改过的第一句。",
)
check("edited", edited["sentence"]["content"] == "改过的第一句。")
still = await call(session, "paragraph_get", paper_id=paper["id"], heading=first)
check("the other sentences kept their ids", still["sentences"][1]["id"] == after["sentences"][1]["id"])
print("\n12. delete one sentence")
before = await call(session, "sentence_list", paper_id=paper["id"])
await call(
session,
"sentence_delete",
paper_id=paper["id"],
sentence_id=still["sentences"][-1]["id"],
)
remaining = await call(session, "sentence_list", paper_id=paper["id"])
check(
"exactly one sentence fewer",
remaining["total"] == before["total"] - 1,
f"{before['total']} -> {remaining['total']}",
)
print("\n13. write a whole generated document in one call")
generated = "\n".join(
[f"# {item['name']}" + "\n" + f"{item['name']}的正文写在标题下面。这句话是第二句。"
for item in outline]
)
whole = await call(
session, "paper_write_text", paper_id=paper["id"], text=generated, split="sentence"
)
check(
"every heading in the document was written",
len(whole["written"]) == len(outline),
json.dumps(whole, ensure_ascii=False),
)
check("nothing was left unmatched", "unmatched" not in whole, json.dumps(whole, ensure_ascii=False))
print("\n14. read the paper as a document")
document = await call(session, "paper_document", paper_id=paper["id"])
check("the text carries every heading", all(item["name"] in document["text"] for item in outline))
check("and every paragraph", document["text"].count("正文写在标题下面") == len(outline))
structured = await call(
session, "paper_document", paper_id=paper["id"], format="json"
)
check(
"the json form carries sentence ids",
all(
row.get("id") is not None
for paragraph in structured["paragraphs"]
for row in paragraph["sentences"]
),
)
check(
"it names the citation style the entries are printed in",
structured["reference_style"] == "springer",
str(structured.get("reference_style")),
)
# Step 13 rewrote every paragraph, so the citation step 9 wrote is gone
# and this paper cites nothing: the 参考文献 list is reported as empty
# rather than left out, because "this cites nothing yet" is an answer.
check(
"and reports the 参考文献 list as empty when nothing is cited",
structured["references"] == [],
json.dumps(structured["references"], ensure_ascii=False),
)
print("\n15. search across papers")
found = await call(session, "sentence_search", keyword="正文写在标题下面", paper_id=paper["id"])
check("the search finds what was written", found["total"] >= 1, str(found["total"]))
print("\n16. the writer asks for a line break")
# A paragraph is its sentences run together, so a formula, a list of
# items or an author line has to say where the lines are. A *display*
# formula does not have to: the paper view sets one on a line of its own
# by definition, and the text a tool hands back has to agree with the
# page the writer is looking at, or the two disagree about the lines.
prose = outline[2]
await call(
session,
"paper_write",
paper_id=paper["id"],
paragraphs=[
{
"heading": prose["name"],
"sentences": [
"本段说明公式的来历。",
{"content": "$$ E = mc^2 $$"},
{"content": "其中 c 为光速。", "line_break_before": True},
],
}
],
)
written = await call(
session, "paragraph_get", paper_id=paper["id"], heading=prose["name"]
)
check(
"the flag is stored per sentence",
[row["line_break_before"] for row in written["sentences"]]
== [False, False, True],
json.dumps(written["sentences"], ensure_ascii=False),
)
check(
"a formula takes its own line unasked, and a flagged sentence too",
written["text"] == "本段说明公式的来历。\n$$ E = mc^2 $$\n其中 c 为光速。",
repr(written["text"]),
)
check(
"and the flag replaces the separator rather than joining it",
"\n " not in written["text"],
repr(written["text"]),
)
whole = await call(session, "paper_document", paper_id=paper["id"])
check(
"the whole document prints it the same way",
"本段说明公式的来历。\n$$ E = mc^2 $$\n其中 c 为光速。" in whole["text"],
repr(whole["text"][-260:]),
)
added = await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=prose["name"],
content="另一条列举。",
line_break_before=True,
)
check(
"sentence_add can ask for one as well",
added["sentence"]["line_break_before"] is True,
json.dumps(added["sentence"], ensure_ascii=False),
)
# Toggled on the sentence that follows the prose, not the one that
# follows the formula: the formula is a block, so the sentence after it
# starts a line whatever its flag says, and a check written on that one
# would pass for the wrong reason.
joined = added["sentence"]["id"]
off = await call(
session,
"sentence_update",
paper_id=paper["id"],
sentence_id=joined,
line_break_before=False,
)
check(
"the break can be taken off again",
off["sentence"]["line_break_before"] is False,
json.dumps(off, ensure_ascii=False),
)
rejoined = await call(
session, "paragraph_get", paper_id=paper["id"], heading=prose["name"]
)
check(
"and the two sentences run on as one line",
"\n另一条列举。" not in rejoined["text"]
and "其中 c 为光速。另一条列举。" in rejoined["text"],
repr(rejoined["text"]),
)
check(
"while the line after the formula stays broken, flag or no flag",
"\n其中 c 为光速。" in rejoined["text"],
repr(rejoined["text"]),
)
await call(
session,
"sentence_update",
paper_id=paper["id"],
sentence_id=joined,
line_break_before=True,
)
relined = await call(
session, "paragraph_get", paper_id=paper["id"], heading=prose["name"]
)
check(
"asking for it again puts the line back",
"\n另一条列举。" in relined["text"],
repr(relined["text"]),
)
moved_texts = [row["content"] for row in written["sentences"]] + [
added["sentence"]["content"]
]
await call(
session,
"paragraph_move",
paper_id=paper["id"],
heading=prose["name"],
target_heading=outline[3]["name"],
)
landed = await call(
session, "paragraph_get", paper_id=paper["id"], heading=outline[3]["name"]
)
# Matched by content rather than by id: a move rewrites the rows, so the
# ids the source paragraph had do not survive it. Citations are rebuilt
# the same way, which is why they come back attached to the new rows.
flag_of = {row["content"]: row["line_break_before"] for row in landed["sentences"]}
check(
"a paragraph carries its line breaks when it moves",
[flag_of.get(text) for text in moved_texts] == [False, False, True, True],
f"moved={moved_texts} reads back as "
f"{[flag_of.get(text) for text in moved_texts]} out of "
f"{json.dumps(landed['sentences'], ensure_ascii=False)}",
)
check(
"and it lands after the target's own sentences, as the tool says",
[row["content"] for row in landed["sentences"]][-len(moved_texts):]
== moved_texts,
json.dumps([row["content"] for row in landed["sentences"]], ensure_ascii=False),
)
print("\n17. the 参考文献 list is printed, not assembled by the client")
# Step 10 deleted the reference this run made, so the bibliography step
# brings its own work to cite. It is cited twice on purpose: one work,
# one entry, one number in both places is the rule being checked.
bibliography = await call(
session,
"reference_create",
reference={
"type": "journal-article",
"title": "SMOKE MCP 参考文献表",
"container_title": "Journal of Smoke Tests",
"publication_year": 2024,
"volume": "12",
"issue": "3",
"pages": "45-67",
"doi": "10.1000/smoke.mcp.002",
"authors": [
{"name": "Smith, John", "family_name": "Smith", "given_name": "John"}
],
},
)
cited_reference = bibliography["id"]
_created_references.append(cited_reference)
first = await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=outline[3]["name"],
content="A cited sentence.",
citations=[{"quote": "引用的原话", "reference_id": cited_reference}],
)
await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=outline[3]["name"],
content="A second sentence, same work.",
line_break_before=True,
citations=[{"quote": "再次引用", "reference_id": cited_reference}],
)
document = await call(
session, "paper_document", paper_id=paper["id"], format="json"
)
check(
"the style the entries are written in is named",
document["reference_style"] == "springer",
str(document.get("reference_style")),
)
check(
"one row per place in the text",
len(document["citations"]) == 2,
json.dumps(document["citations"], ensure_ascii=False),
)
check(
"but one entry for the work, however often it is cited",
len(document["references"]) == 1,
json.dumps(document["references"], ensure_ascii=False),
)
entry = document["references"][0]
check(
"the entry is a bibliography line rather than the quoted words",
entry["text"].startswith("Smith, J.: SMOKE MCP 参考文献表")
and "Journal of Smoke Tests 12(3), 45–67 (2024)" in entry["text"]
and entry["text"].endswith("https://doi.org/10.1000/smoke.mcp.002"),
entry["text"],
)
check(
"a complete entry is reported as complete",
entry["missing"] == [],
str(entry["missing"]),
)
check(
"both places wear the one number the entry has",
{row["index"] for row in document["citations"]} == {entry["index"]},
json.dumps(document["citations"], ensure_ascii=False),
)
check(
"and the entry names both of them",
len(entry["citations"]) == 2
and first["sentence"]["id"]
in {row["sentence_id"] for row in document["citations"]},
json.dumps(entry, ensure_ascii=False),
)
as_text = await call(session, "paper_document", paper_id=paper["id"])
check(
"the text form carries the 参考文献 block with that entry",
f"参考文献\n[{entry['index']}] {entry['text']}" in as_text["text"],
repr(as_text["text"][-300:]),
)
check(
"and marks both places with that same number",
f"A cited sentence.[{entry['index']}]" in as_text["text"]
and f"A second sentence, same work.[{entry['index']}]" in as_text["text"],
repr(as_text["text"][-700:]),
)
check(
"and the bibliography lists the work exactly once",
len(
[
line
for line in as_text["text"].splitlines()
if line.startswith(f"[{entry['index']}] ")
]
)
== 1,
repr(as_text["text"][-300:]),
)
print("\n18. a figure goes in, and prints on a line of its own")
# The bytes go to the object store and the sentence keeps the address,
# so this is the one place the suite needs MinIO as well as the database.
# The 1x1 GIF is the smallest real picture there is, which is all that is
# needed to prove the bytes arrive and the address comes back.
figure = await call(
session,
"image_upload",
data_base64=f"data:image/gif;base64,{GIF_1PX_BASE64}",
caption="图 1 试验装置示意图",
width="70%",
)
check(
"the key is the shape the store writes, named after the bytes",
re.fullmatch(r"\d{4}/\d{2}/[0-9a-f]{16}\.gif", figure["key"]) is not None,
figure["key"],
)
check(
"the URL is same-origin under the API prefix",
figure["url"] == f"/api/images/{figure['key']}",
figure["url"],
)
check(
"the markdown is the sentence's content, caption and size included",
figure["markdown"]
== f"![图 1 试验装置示意图](/api/images/{figure['key']}){{width=70%}}",
figure["markdown"],
)
check(
"and the sentence that shows it is ready to write as it stands",
figure["sentence"]
== {
"content": figure["markdown"],
"line_break_before": True,
"formula_align": "center",
},
json.dumps(figure["sentence"], ensure_ascii=False),
)
same = await call(
session, "image_upload", data_base64=GIF_1PX_BASE64, filename="另一张.png"
)
check(
"the same picture again is the same object, not a second copy",
same["key"] == figure["key"],
f"{same['key']} vs {figure['key']}",
)
check(
"and the caption defaults to the filename's stem",
same["markdown"] == f"![另一张](/api/images/{figure['key']})",
same["markdown"],
)
# The returned sentence object is written as it came back, unchanged:
# that is the whole contract of the tool.
await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=outline[3]["name"],
**figure["sentence"],
)
with_figure = await call(
session, "paragraph_get", paper_id=paper["id"], heading=outline[3]["name"]
)
check(
"the figure is stored as the sentence it was handed",
with_figure["sentences"][-1]["content"] == figure["markdown"]
and with_figure["sentences"][-1]["line_break_before"] is True,
json.dumps(with_figure["sentences"][-1], ensure_ascii=False),
)
check(
"and the paragraph prints it alone on its own line",
with_figure["text"].endswith("\n" + figure["markdown"]),
repr(with_figure["text"][-200:]),
)
inline_url = same["url"]
inline = f"试验装置如 ![示意图]({inline_url}) 所示。"
await call(
session,
"sentence_add",
paper_id=paper["id"],
heading=outline[3]["name"],
content=inline,
)
with_inline = await call(
session, "paragraph_get", paper_id=paper["id"], heading=outline[3]["name"]
)
check(
"an image inside a sentence stays on the line with the words",
f"\n{inline}" in with_inline["text"]
and f"![示意图]({inline_url})\n" not in with_inline["text"],
repr(with_inline["text"][-260:]),
)
message = await expect_error(
session,
"image_upload",
data_base64=NOT_A_PICTURE_BASE64,
filename="图.svg",
)
check(
"a file that is not a picture is refused, naming the formats",
"PNG" in message and "SVG" in message,
message,
)
tolerant = await call(
session,
"image_upload",
data_base64=GIF_1PX_BASE64,
caption="写法宽容",
width="{w = 400 px}",
)
check(
"how the size is written is forgiving, what it means is not",
tolerant["markdown"] == f"![写法宽容](/api/images/{tolerant['key']}){{width=400px}}",
tolerant["markdown"],
)
message = await expect_error(
session, "image_upload", data_base64=GIF_1PX_BASE64, width="70em"
)
check(
"a size the reader cannot parse is refused rather than quietly ignored",
"70%" in message,
message,
)
message = await expect_error(
session, "image_upload", data_base64=GIF_1PX_BASE64, width="150%"
)
check(
"and so is one that parses but the page would drop on the floor",
"1 到 100" in message,
message,
)
message = await expect_error(
session, "image_upload", data_base64="这不是 base64"
)
check(
"and a value that is not base64 says what to send instead",
"base64" in message,
message,
)
print("\n19. a formula says how it is set on its line")
derivation = outline[2]
await call(
session,
"paper_write",
paper_id=paper["id"],
paragraphs=[
{
"heading": derivation["name"],
"sentences": [
"推导如下。",
{
"content": "$$ a^2 + b^2 = c^2 $$",
"line_break_before": True,
"formula_align": "right",
},
],
}
],
)
aligned = await call(
session, "paragraph_get", paper_id=paper["id"], heading=derivation["name"]
)
check(
"the alignment is stored on the sentence that carries the formula",
[row["formula_align"] for row in aligned["sentences"]]
== ["center", "right"],
json.dumps(aligned["sentences"], ensure_ascii=False),
)
turned = await call(
session,
"sentence_update",
paper_id=paper["id"],
sentence_id=aligned["sentences"][-1]["id"],
formula_align="left",
)
check(
"and it can be changed without touching the formula",
turned["sentence"]["formula_align"] == "left"
and turned["sentence"]["content"] == "$$ a^2 + b^2 = c^2 $$",
json.dumps(turned["sentence"], ensure_ascii=False),
)
await call(
session,
"paragraph_move",
paper_id=paper["id"],
heading=derivation["name"],
target_heading=outline[3]["name"],
)
carried = await call(
session, "paragraph_get", paper_id=paper["id"], heading=outline[3]["name"]
)
check(
"and it travels with the sentence when the paragraph moves",
any(
row["content"] == "$$ a^2 + b^2 = c^2 $$"
and row["formula_align"] == "left"
for row in carried["sentences"]
),
json.dumps(carried["sentences"], ensure_ascii=False),
)
print("\n20. move a paragraph, then clear the one it moved onto")
home = outline[0]
target = outline[1]
relocation = await call(
session,
"paragraph_move",
paper_id=paper["id"],
heading=home["name"],
target_heading=target["name"],
)
check(
"the move reports both ends",
relocation["from"]["position"] == home["position"]
and relocation["to"]["position"] == target["position"],
json.dumps(relocation, ensure_ascii=False),
)
emptied = await call(session, "paragraph_get", paper_id=paper["id"], position=home["position"])
check("the source paragraph is empty now", emptied["sentences"] == [], json.dumps(emptied, ensure_ascii=False))
merged = await call(session, "paragraph_get", paper_id=paper["id"], position=target["position"])
check(
"the target holds both paragraphs' sentences",
len(merged["sentences"]) == 4,
json.dumps(merged["sentences"], ensure_ascii=False),
)
cleared = await call(
session, "paragraph_delete", paper_id=paper["id"], position=target["position"]
)
check("clearing reports how much it removed", cleared["deleted_sentences"] == 4, json.dumps(cleared, ensure_ascii=False))
after_clear = await call(session, "paragraph_get", paper_id=paper["id"], position=target["position"])
check("and the paragraph is empty", after_clear["sentences"] == [], json.dumps(after_clear, ensure_ascii=False))
print("\n21. the field library round-trips")
field = await call(session, "field_create", name="SMOKE MCP 字段", level=2, font_size=9.5)
check("created with its typography", field["level"] == 2 and field["font_size"] == 9.5, json.dumps(field, ensure_ascii=False))
renamed = await call(session, "field_update", field_id=field["id"], name="SMOKE MCP 字段(改)")
check("renamed", renamed["name"] == "SMOKE MCP 字段(改)", json.dumps(renamed, ensure_ascii=False))
dropped = await call(session, "field_delete", field_ids=field["id"])
check("deleted while no template places it", dropped["deleted"] == 1, json.dumps(dropped, ensure_ascii=False))
check("and it is reported as found, not missing", dropped["missing"] == [], json.dumps(dropped, ensure_ascii=False))
print("\n22. switch the template and keep the content")
spare = await call(
session,
"template_create",
name="SMOKE MCP 模板",
abstract="临时模板",
field_ids=[_field_id(outline[0]["name"])],
)
_created_templates.append(spare["id"])
check("a template can be built from field ids", spare["fields"][0]["position"] == 10, json.dumps(spare, ensure_ascii=False))
rebuilt = await call(
session,
"template_update",
template_id=spare["id"],
name="SMOKE MCP 模板",
field_ids=[_field_id(outline[0]["name"]), _field_id(outline[1]["name"])],
)
check(
"the outline is replaced wholesale and renumbered",
[item["position"] for item in rebuilt["fields"]] == [10, 20],
json.dumps(rebuilt, ensure_ascii=False),
)
sentences_before = (await call(session, "sentence_list", paper_id=paper["id"]))["total"]
switched = await call(
session, "paper_update", paper_id=paper["id"], template_id=spare["id"]
)
check("the switch is reported", switched["template_changed"] is True)
switched_document = await call(
session, "paper_document", paper_id=paper["id"], format="json"
)
check(
"content survives the switch, under 未设定",
sum(len(row["sentences"]) for row in switched_document["paragraphs"])
== sentences_before,
json.dumps(switched_document, ensure_ascii=False)[:400],
)
check(
"positions the new template does not define are marked unmatched",
any(row["matched"] is False for row in switched_document["paragraphs"]),
json.dumps(switched_document, ensure_ascii=False)[:400],
)
print("\n23. a template in use cannot be deleted")
message = await expect_error(session, "template_delete", template_ids=spare["id"])
check("the refusal names the blocker", "SMOKE MCP 模板" in message, message)
print("\n24. deleting a paper needs an explicit confirmation")
message = await expect_error(session, "paper_delete", paper_ids=paper["id"])
check("the refusal explains itself", "confirm=true" in message, message)
print("\n25. delete the paper and the template")
deleted = await call(session, "paper_delete", paper_ids=paper["id"], confirm=True)
check("deleted", deleted["deleted"] == 1, json.dumps(deleted, ensure_ascii=False))
_created_papers.remove(paper["id"])
gone = await call(session, "template_delete", template_ids=spare["id"])
check("the template is free to go now", gone["deleted"] == 1, json.dumps(gone, ensure_ascii=False))
_created_templates.remove(spare["id"])
print(f"\nall {_checks} checks passed")
return 0
def _field_id(name: str) -> int:
"""The library field behind a heading name, for the template-create step."""
from app.models import TemplateTitle
with SessionLocal() as db:
row = db.scalars(
select(TemplateTitle).where(TemplateTitle.name == name)
).first()
if row is None:
raise AssertionError(f"field {name!r} not in the library")
return row.id
def cleanup() -> None:
"""Remove anything this run created, even after a failed expectation."""
if _created_papers:
with SessionLocal() as db:
for paper in db.scalars(select(Paper).where(Paper.id.in_(_created_papers))).all():
db.delete(paper)
db.commit()
print(f"cleaned up papers {_created_papers}")
if _created_templates:
with SessionLocal() as db:
for template in db.scalars(
select(Template).where(Template.id.in_(_created_templates))
).all():
db.delete(template)
db.commit()
print(f"cleaned up templates {_created_templates}")
# References go before tags: a reference carries tag links, and a tag
# deleted out from under one would leave the link pointing at nothing.
if _created_references:
with SessionLocal() as db:
for reference in db.scalars(
select(Reference).where(Reference.id.in_(_created_references))
).all():
db.delete(reference)
db.commit()
print(f"cleaned up references {_created_references}")
if _created_tags:
with SessionLocal() as db:
for tag in db.scalars(
select(ReferenceTag).where(ReferenceTag.id.in_(_created_tags))
).all():
db.delete(tag)
db.commit()
print(f"cleaned up tags {_created_tags}")
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
"""``--url`` switches the suite from spawning a server to dialling one."""
parser = argparse.ArgumentParser(description="paper-doc MCP 冒烟测试")
parser.add_argument(
"--url",
default=os.getenv("MCP_SMOKE_URL"),
help="连到已在运行的 streamable-http 服务,例如 http://127.0.0.1:8931/mcp;"
"不传则自己拉起 stdio 子进程",
)
parser.add_argument(
"--token",
default=os.getenv("MCP_HTTP_TOKEN"),
help="HTTP 传输的 Bearer 口令",
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
# The smoke test writes real rows, so it removes its own first: a previous
# run that died before cleanup would otherwise leave a duplicate title.
with SessionLocal() as db:
stale = db.scalars(
select(Paper).where(or_(Paper.title == "SMOKE MCP 论文"))
).all()
for paper in stale:
db.delete(paper)
stale_templates = db.scalars(
select(Template).where(Template.name == "SMOKE MCP 模板")
).all()
for template in stale_templates:
db.delete(template)
# References and tags from a run that died before cleanup: the suite
# creates them through the tools now, so a failure between the two
# steps leaves rows the next run would trip over (a duplicate tag name
# refuses the create outright).
for reference in db.scalars(
select(Reference).where(Reference.title.like("SMOKE MCP%"))
).all():
db.delete(reference)
db.flush()
for tag in db.scalars(
select(ReferenceTag).where(ReferenceTag.name.like("SMOKE MCP%"))
).all():
db.delete(tag)
db.commit()
try:
return anyio.run(run, args.url, args.token)
except AssertionError as error:
print(f"\nFAILED: {error}", file=sys.stderr)
return 1
finally:
cleanup()
if __name__ == "__main__":
sys.exit(main())