Files
paper-doc/backend/app/schemas/reference.py
T
govin d7bb777264 backend: add the reference library (参考文献)
One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
2026-09-22 17:13:32 +08:00

407 lines
14 KiB
Python

"""Request/response schemas for the reference library (参考文献).
The field set and where it comes from is documented on
:mod:`app.models.reference`; this module is about the wire format:
* **Nothing is stored that a style can derive.** ``authors`` arrives as an
ordered list and replaces the stored list wholesale, like a template's
placements — the citation prints them in exactly the order sent.
* **A DOI is stored canonically.** ``https://doi.org/10.1000/xyz``,
``doi:10.1000/xyz`` and ``10.1000/xyz`` are the same DOI, so the prefix is
stripped on write and the column holds one spelling.
* **A date is as precise as it is known.** Year, month and day are separate and
independently optional, which is what CSL calls a partial date: a book has a
year, a newspaper article a day, a patent its 公告日期.
"""
from datetime import date, datetime
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from app.models.reference import (
REFERENCE_AUTHOR_ROLES,
REFERENCE_LANGUAGES,
REFERENCE_TYPES,
)
#: Prefixes people paste along with a DOI. Stripped on write so the column has
#: one spelling, and so a citation builder never has to test for them.
_DOI_PREFIXES = (
"https://doi.org/",
"http://doi.org/",
"https://dx.doi.org/",
"http://dx.doi.org/",
"doi:",
"DOI:",
)
def normalize_doi(value: str | None) -> str | None:
"""Return the bare DOI, or ``None`` when nothing was given."""
if value is None:
return None
text = value.strip()
for prefix in _DOI_PREFIXES:
if text.lower().startswith(prefix.lower()):
text = text[len(prefix) :]
break
return text.strip() or None
class ReferenceAuthorInput(BaseModel):
"""One contributor as the client sends it.
``name`` is required — it is what prints. ``family_name`` / ``given_name``
are optional enrichment for styles that invert or initialise a name, and a
``sort`` that is left out is assigned from the position in the list.
"""
name: str = Field(min_length=1, max_length=255)
family_name: str | None = Field(default=None, max_length=128)
given_name: str | None = Field(default=None, max_length=128)
role: str = Field(default="author", max_length=16)
sort: int | None = Field(default=None, ge=-1_000_000, le=1_000_000)
@field_validator("name")
@classmethod
def _require_name(cls, value: str) -> str:
text = value.strip()
if not text:
raise ValueError("责任者姓名不能为空")
return text
@field_validator("family_name", "given_name")
@classmethod
def _strip_optional(cls, value: str | None) -> str | None:
if value is None:
return None
return value.strip() or None
@field_validator("role")
@classmethod
def _check_role(cls, value: str) -> str:
if value not in REFERENCE_AUTHOR_ROLES:
raise ValueError(
"责任者角色只能是:" + "、".join(REFERENCE_AUTHOR_ROLES)
)
return value
class ReferenceAuthorRead(BaseModel):
"""A stored contributor."""
model_config = ConfigDict(from_attributes=True)
id: int
sort: int
name: str
family_name: str | None
given_name: str | None
role: str
class ReferenceBase(BaseModel):
"""Every field a reference may carry, all optional but the title."""
type: str = Field(default="journal-article", max_length=32)
title: str = Field(min_length=1, max_length=512)
subtitle: str | None = Field(default=None, max_length=512)
title_translated: str | None = Field(default=None, max_length=512)
title_short: str | None = Field(default=None, max_length=255)
container_title: str | None = Field(default=None, max_length=512)
container_title_short: str | None = Field(default=None, max_length=255)
series_title: str | None = Field(default=None, max_length=512)
series_number: str | None = Field(default=None, max_length=32)
edition: str | None = Field(default=None, max_length=64)
volume: str | None = Field(default=None, max_length=32)
issue: str | None = Field(default=None, max_length=32)
pages: str | None = Field(default=None, max_length=64)
article_number: str | None = Field(default=None, max_length=64)
publisher: str | None = Field(default=None, max_length=255)
publisher_place: str | None = Field(default=None, max_length=255)
event_title: str | None = Field(default=None, max_length=512)
event_place: str | None = Field(default=None, max_length=255)
event_date: str | None = Field(default=None, max_length=64)
publication_year: int | None = Field(default=None, ge=1, le=2999)
publication_month: int | None = Field(default=None, ge=1, le=12)
publication_day: int | None = Field(default=None, ge=1, le=31)
accessed_date: date | None = None
doi: str | None = Field(default=None, max_length=255)
isbn: str | None = Field(default=None, max_length=32)
issn: str | None = Field(default=None, max_length=32)
url: str | None = Field(default=None, max_length=1024)
patent_country: str | None = Field(default=None, max_length=64)
patent_number: str | None = Field(default=None, max_length=64)
standard_number: str | None = Field(default=None, max_length=64)
report_number: str | None = Field(default=None, max_length=64)
arxiv_id: str | None = Field(default=None, max_length=64)
abstract: str | None = None
keywords: str | None = Field(default=None, max_length=512)
note: str | None = None
citation_key: str | None = Field(default=None, max_length=64)
language: str | None = Field(default=None, max_length=16)
authors: list[ReferenceAuthorInput] = Field(default_factory=list)
@field_validator("title")
@classmethod
def _require_title(cls, value: str) -> str:
text = value.strip()
if not text:
raise ValueError("文献题名不能为空")
return text
@field_validator("type")
@classmethod
def _check_type(cls, value: str) -> str:
if value not in REFERENCE_TYPES:
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
return value
@field_validator("language")
@classmethod
def _check_language(cls, value: str | None) -> str | None:
if value is None or value == "":
return None
if value not in REFERENCE_LANGUAGES:
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
return value
@field_validator("doi")
@classmethod
def _normalise_doi(cls, value: str | None) -> str | None:
return normalize_doi(value)
@field_validator(
"subtitle",
"title_translated",
"title_short",
"container_title",
"container_title_short",
"series_title",
"series_number",
"edition",
"volume",
"issue",
"pages",
"article_number",
"publisher",
"publisher_place",
"event_title",
"event_place",
"event_date",
"isbn",
"issn",
"url",
"patent_country",
"patent_number",
"standard_number",
"report_number",
"arxiv_id",
"keywords",
"citation_key",
)
@classmethod
def _blank_to_none(cls, value: str | None) -> str | None:
"""Treat a whitespace-only string as "not set" rather than as a value."""
if value is None:
return None
return value.strip() or None
@model_validator(mode="after")
def _check_online_resource(self) -> "ReferenceBase":
"""An online resource must say where it is.
GB/T 7714 prints a URL for ``[EB/OL]`` and nothing else identifies the
document; a row typed as a web page or a dataset with no URL is a row
that cannot be cited.
"""
if self.type in ("web-page", "dataset") and not (self.url or self.doi):
raise ValueError("网页/数据集类型的文献必须至少给出 URL 或 DOI")
return self
class ReferenceCreate(ReferenceBase):
"""Payload for ``POST /references``."""
class ReferenceUpdate(BaseModel):
"""Payload for ``PATCH /references/{id}`` — every part optional.
``authors`` is a full replacement when present, like a template's
placements: the citation prints the list in the order it arrives.
"""
type: str | None = Field(default=None, max_length=32)
title: str | None = Field(default=None, min_length=1, max_length=512)
subtitle: str | None = Field(default=None, max_length=512)
title_translated: str | None = Field(default=None, max_length=512)
title_short: str | None = Field(default=None, max_length=255)
container_title: str | None = Field(default=None, max_length=512)
container_title_short: str | None = Field(default=None, max_length=255)
series_title: str | None = Field(default=None, max_length=512)
series_number: str | None = Field(default=None, max_length=32)
edition: str | None = Field(default=None, max_length=64)
volume: str | None = Field(default=None, max_length=32)
issue: str | None = Field(default=None, max_length=32)
pages: str | None = Field(default=None, max_length=64)
article_number: str | None = Field(default=None, max_length=64)
publisher: str | None = Field(default=None, max_length=255)
publisher_place: str | None = Field(default=None, max_length=255)
event_title: str | None = Field(default=None, max_length=512)
event_place: str | None = Field(default=None, max_length=255)
event_date: str | None = Field(default=None, max_length=64)
publication_year: int | None = Field(default=None, ge=1, le=2999)
publication_month: int | None = Field(default=None, ge=1, le=12)
publication_day: int | None = Field(default=None, ge=1, le=31)
accessed_date: date | None = None
doi: str | None = Field(default=None, max_length=255)
isbn: str | None = Field(default=None, max_length=32)
issn: str | None = Field(default=None, max_length=32)
url: str | None = Field(default=None, max_length=1024)
patent_country: str | None = Field(default=None, max_length=64)
patent_number: str | None = Field(default=None, max_length=64)
standard_number: str | None = Field(default=None, max_length=64)
report_number: str | None = Field(default=None, max_length=64)
arxiv_id: str | None = Field(default=None, max_length=64)
abstract: str | None = None
keywords: str | None = Field(default=None, max_length=512)
note: str | None = None
citation_key: str | None = Field(default=None, max_length=64)
language: str | None = Field(default=None, max_length=16)
authors: list[ReferenceAuthorInput] | None = None
@field_validator("title")
@classmethod
def _require_title(cls, value: str | None) -> str | None:
if value is None:
return None
text = value.strip()
if not text:
raise ValueError("文献题名不能为空")
return text
@field_validator("type")
@classmethod
def _check_type(cls, value: str | None) -> str | None:
if value is None:
return None
if value not in REFERENCE_TYPES:
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
return value
@field_validator("language")
@classmethod
def _check_language(cls, value: str | None) -> str | None:
if value is None or value == "":
return None
if value not in REFERENCE_LANGUAGES:
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
return value
@field_validator("doi")
@classmethod
def _normalise_doi(cls, value: str | None) -> str | None:
return normalize_doi(value)
class ReferenceListItem(BaseModel):
"""A reference as it appears in the library table."""
id: int
type: str
title: str
subtitle: str | None
container_title: str | None
publication_year: int | None
volume: str | None
issue: str | None
pages: str | None
doi: str | None
#: The printed author list, joined — a table row needs the line, not the
#: rows behind it.
authors_line: str
author_count: int
#: How many sentences cite it. Drives the refusal message on delete, and
#: tells the writer at a glance what is already used.
citation_count: int
created_at: datetime
updated_at: datetime
class ReferenceRead(BaseModel):
"""A reference with everything the table does not need."""
id: int
type: str
title: str
subtitle: str | None
title_translated: str | None
title_short: str | None
container_title: str | None
container_title_short: str | None
series_title: str | None
series_number: str | None
edition: str | None
volume: str | None
issue: str | None
pages: str | None
article_number: str | None
publisher: str | None
publisher_place: str | None
event_title: str | None
event_place: str | None
event_date: str | None
publication_year: int | None
publication_month: int | None
publication_day: int | None
accessed_date: date | None
doi: str | None
isbn: str | None
issn: str | None
url: str | None
patent_country: str | None
patent_number: str | None
standard_number: str | None
report_number: str | None
arxiv_id: str | None
abstract: str | None
keywords: str | None
note: str | None
citation_key: str | None
language: str | None
authors: list[ReferenceAuthorRead]
citation_count: int
created_at: datetime
updated_at: datetime
class ReferenceCitationRead(BaseModel):
"""One sentence that cites a reference — the reverse lookup.
"Which of my sentences use this article?" is the question a reference
library exists to answer, and it needs the sentence, the paragraph it sits
in, and the paper it belongs to.
"""
citation_id: int
sentence_id: int
sentence_content: str
quote: str
paper_id: int
paper_title: str
paper_template_filed_sort: int
paragraph_name: str | None = None
__all__ = [
"ReferenceAuthorInput",
"ReferenceAuthorRead",
"ReferenceBase",
"ReferenceCitationRead",
"ReferenceCreate",
"ReferenceListItem",
"ReferenceRead",
"ReferenceUpdate",
"normalize_doi",
]