backend: add the reference library (参考文献)

One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
This commit is contained in:
2026-09-22 17:13:32 +08:00
parent 103b5544ba
commit d7bb777264
18 changed files with 1966 additions and 16 deletions
+50 -4
View File
@@ -26,8 +26,8 @@ import argparse
import json
import os
import sys
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from collections.abc import AsyncIterator, Iterator
from contextlib import asynccontextmanager, contextmanager
from pathlib import Path
from typing import Any
@@ -43,7 +43,7 @@ from mcp.client.stdio import StdioServerParameters, stdio_client # noqa: E402
from sqlalchemy import or_, select # noqa: E402
from app.db.session import SessionLocal # noqa: E402
from app.models import Paper, Template # noqa: E402
from app.models import Paper, Reference, Template # noqa: E402
SERVER_SCRIPT = BACKEND_DIR / "scripts" / "mcp_server.py"
@@ -97,6 +97,15 @@ async def expect_error(session: ClientSession, tool: str, **arguments: Any) -> s
)
class _Box:
"""A one-slot holder, so the citation helper can read what the context set."""
value: int | None = None
_CITATION_REFERENCE_ID = _Box()
def pick_template() -> tuple[int, list[dict[str, Any]]]:
"""The template with the most titles, and its outline as the tools see it."""
with SessionLocal() as db:
@@ -145,6 +154,11 @@ async def connect(url: str | None, token: str | None) -> AsyncIterator[ClientSes
async def run(url: str | None = None, token: str | None = None) -> int:
with reference_for_citation():
return await _run(url, token)
async def _run(url: str | None = None, token: str | None = None) -> int:
template_id, outline = pick_template()
check("a template with fields exists", len(outline) >= 3, f"got {len(outline)}")
first = outline[0]["name"]
@@ -257,7 +271,7 @@ async def run(url: str | None = None, token: str | None = None) -> int:
paper_id=paper["id"],
heading=first,
content="A cited sentence.",
citations=[{"quote": "引用的原话", "reference_id": 7}],
citations=[{"quote": "引用的原话", "reference_id": _reference_id()}],
)
check("appended", added["sentence"]["content"] == "A cited sentence.")
check("the citation is stored", added["sentence"]["citations"][0]["quote"] == "引用的原话")
@@ -436,6 +450,38 @@ async def run(url: str | None = None, token: str | None = None) -> int:
return 0
@contextmanager
def reference_for_citation() -> Iterator[int]:
"""A library entry the citation checks can point at.
A citation names real literature now, so the test has to have some: the MCP
surface has no reference tools yet, and this creates the row directly.
"""
with SessionLocal() as db:
reference = Reference(type="journal-article", title="SMOKE MCP 文献")
db.add(reference)
db.commit()
db.refresh(reference)
reference_id = reference.id
_CITATION_REFERENCE_ID.value = reference_id
try:
yield reference_id
finally:
with SessionLocal() as db:
stored = db.get(Reference, reference_id)
if stored is not None:
db.delete(stored)
db.commit()
_CITATION_REFERENCE_ID.value = None
def _reference_id() -> int:
"""The id the current citation check should use."""
if _CITATION_REFERENCE_ID.value is None:
raise AssertionError("no reference in scope for this citation check")
return _CITATION_REFERENCE_ID.value
def _field_id(name: str) -> int:
"""The library field behind a heading name, for the template-create step."""
from app.models import TemplateTitle