backend: add the reference library (参考文献)
One table per document the writer has read, and one for its contributors:
reference id, type, title, subtitle, title_translated,
container_title, volume, issue, pages, publisher,
publisher_place, event_*, publication_year/month/day,
accessed_date, doi, isbn, issn, url, patent_*,
standard_number, report_number, arxiv_id, abstract,
keywords, note, citation_key, language
reference_author reference_id, sort, name, family_name, given_name, role
The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:
* **Partial dates.** Year, month and day are separate nullable integers, which
is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
"S. Zhang"), so the printed form and its two halves are stored separately —
display reads `name`, a reformatting style reads the halves, and an
institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
`S1` and `e12345`, and a citation reproduces what is printed.
The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.
Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.
API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.
Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
This commit is contained in:
@@ -0,0 +1,406 @@
|
||||
"""Request/response schemas for the reference library (参考文献).
|
||||
|
||||
The field set and where it comes from is documented on
|
||||
:mod:`app.models.reference`; this module is about the wire format:
|
||||
|
||||
* **Nothing is stored that a style can derive.** ``authors`` arrives as an
|
||||
ordered list and replaces the stored list wholesale, like a template's
|
||||
placements — the citation prints them in exactly the order sent.
|
||||
* **A DOI is stored canonically.** ``https://doi.org/10.1000/xyz``,
|
||||
``doi:10.1000/xyz`` and ``10.1000/xyz`` are the same DOI, so the prefix is
|
||||
stripped on write and the column holds one spelling.
|
||||
* **A date is as precise as it is known.** Year, month and day are separate and
|
||||
independently optional, which is what CSL calls a partial date: a book has a
|
||||
year, a newspaper article a day, a patent its 公告日期.
|
||||
"""
|
||||
|
||||
from datetime import date, datetime
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||
|
||||
from app.models.reference import (
|
||||
REFERENCE_AUTHOR_ROLES,
|
||||
REFERENCE_LANGUAGES,
|
||||
REFERENCE_TYPES,
|
||||
)
|
||||
|
||||
#: Prefixes people paste along with a DOI. Stripped on write so the column has
|
||||
#: one spelling, and so a citation builder never has to test for them.
|
||||
_DOI_PREFIXES = (
|
||||
"https://doi.org/",
|
||||
"http://doi.org/",
|
||||
"https://dx.doi.org/",
|
||||
"http://dx.doi.org/",
|
||||
"doi:",
|
||||
"DOI:",
|
||||
)
|
||||
|
||||
|
||||
def normalize_doi(value: str | None) -> str | None:
|
||||
"""Return the bare DOI, or ``None`` when nothing was given."""
|
||||
if value is None:
|
||||
return None
|
||||
text = value.strip()
|
||||
for prefix in _DOI_PREFIXES:
|
||||
if text.lower().startswith(prefix.lower()):
|
||||
text = text[len(prefix) :]
|
||||
break
|
||||
return text.strip() or None
|
||||
|
||||
|
||||
class ReferenceAuthorInput(BaseModel):
|
||||
"""One contributor as the client sends it.
|
||||
|
||||
``name`` is required — it is what prints. ``family_name`` / ``given_name``
|
||||
are optional enrichment for styles that invert or initialise a name, and a
|
||||
``sort`` that is left out is assigned from the position in the list.
|
||||
"""
|
||||
|
||||
name: str = Field(min_length=1, max_length=255)
|
||||
family_name: str | None = Field(default=None, max_length=128)
|
||||
given_name: str | None = Field(default=None, max_length=128)
|
||||
role: str = Field(default="author", max_length=16)
|
||||
sort: int | None = Field(default=None, ge=-1_000_000, le=1_000_000)
|
||||
|
||||
@field_validator("name")
|
||||
@classmethod
|
||||
def _require_name(cls, value: str) -> str:
|
||||
text = value.strip()
|
||||
if not text:
|
||||
raise ValueError("责任者姓名不能为空")
|
||||
return text
|
||||
|
||||
@field_validator("family_name", "given_name")
|
||||
@classmethod
|
||||
def _strip_optional(cls, value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
return value.strip() or None
|
||||
|
||||
@field_validator("role")
|
||||
@classmethod
|
||||
def _check_role(cls, value: str) -> str:
|
||||
if value not in REFERENCE_AUTHOR_ROLES:
|
||||
raise ValueError(
|
||||
"责任者角色只能是:" + "、".join(REFERENCE_AUTHOR_ROLES)
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
class ReferenceAuthorRead(BaseModel):
|
||||
"""A stored contributor."""
|
||||
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: int
|
||||
sort: int
|
||||
name: str
|
||||
family_name: str | None
|
||||
given_name: str | None
|
||||
role: str
|
||||
|
||||
|
||||
class ReferenceBase(BaseModel):
|
||||
"""Every field a reference may carry, all optional but the title."""
|
||||
|
||||
type: str = Field(default="journal-article", max_length=32)
|
||||
title: str = Field(min_length=1, max_length=512)
|
||||
subtitle: str | None = Field(default=None, max_length=512)
|
||||
title_translated: str | None = Field(default=None, max_length=512)
|
||||
title_short: str | None = Field(default=None, max_length=255)
|
||||
container_title: str | None = Field(default=None, max_length=512)
|
||||
container_title_short: str | None = Field(default=None, max_length=255)
|
||||
series_title: str | None = Field(default=None, max_length=512)
|
||||
series_number: str | None = Field(default=None, max_length=32)
|
||||
edition: str | None = Field(default=None, max_length=64)
|
||||
volume: str | None = Field(default=None, max_length=32)
|
||||
issue: str | None = Field(default=None, max_length=32)
|
||||
pages: str | None = Field(default=None, max_length=64)
|
||||
article_number: str | None = Field(default=None, max_length=64)
|
||||
publisher: str | None = Field(default=None, max_length=255)
|
||||
publisher_place: str | None = Field(default=None, max_length=255)
|
||||
event_title: str | None = Field(default=None, max_length=512)
|
||||
event_place: str | None = Field(default=None, max_length=255)
|
||||
event_date: str | None = Field(default=None, max_length=64)
|
||||
publication_year: int | None = Field(default=None, ge=1, le=2999)
|
||||
publication_month: int | None = Field(default=None, ge=1, le=12)
|
||||
publication_day: int | None = Field(default=None, ge=1, le=31)
|
||||
accessed_date: date | None = None
|
||||
doi: str | None = Field(default=None, max_length=255)
|
||||
isbn: str | None = Field(default=None, max_length=32)
|
||||
issn: str | None = Field(default=None, max_length=32)
|
||||
url: str | None = Field(default=None, max_length=1024)
|
||||
patent_country: str | None = Field(default=None, max_length=64)
|
||||
patent_number: str | None = Field(default=None, max_length=64)
|
||||
standard_number: str | None = Field(default=None, max_length=64)
|
||||
report_number: str | None = Field(default=None, max_length=64)
|
||||
arxiv_id: str | None = Field(default=None, max_length=64)
|
||||
abstract: str | None = None
|
||||
keywords: str | None = Field(default=None, max_length=512)
|
||||
note: str | None = None
|
||||
citation_key: str | None = Field(default=None, max_length=64)
|
||||
language: str | None = Field(default=None, max_length=16)
|
||||
authors: list[ReferenceAuthorInput] = Field(default_factory=list)
|
||||
|
||||
@field_validator("title")
|
||||
@classmethod
|
||||
def _require_title(cls, value: str) -> str:
|
||||
text = value.strip()
|
||||
if not text:
|
||||
raise ValueError("文献题名不能为空")
|
||||
return text
|
||||
|
||||
@field_validator("type")
|
||||
@classmethod
|
||||
def _check_type(cls, value: str) -> str:
|
||||
if value not in REFERENCE_TYPES:
|
||||
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
|
||||
return value
|
||||
|
||||
@field_validator("language")
|
||||
@classmethod
|
||||
def _check_language(cls, value: str | None) -> str | None:
|
||||
if value is None or value == "":
|
||||
return None
|
||||
if value not in REFERENCE_LANGUAGES:
|
||||
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
|
||||
return value
|
||||
|
||||
@field_validator("doi")
|
||||
@classmethod
|
||||
def _normalise_doi(cls, value: str | None) -> str | None:
|
||||
return normalize_doi(value)
|
||||
|
||||
@field_validator(
|
||||
"subtitle",
|
||||
"title_translated",
|
||||
"title_short",
|
||||
"container_title",
|
||||
"container_title_short",
|
||||
"series_title",
|
||||
"series_number",
|
||||
"edition",
|
||||
"volume",
|
||||
"issue",
|
||||
"pages",
|
||||
"article_number",
|
||||
"publisher",
|
||||
"publisher_place",
|
||||
"event_title",
|
||||
"event_place",
|
||||
"event_date",
|
||||
"isbn",
|
||||
"issn",
|
||||
"url",
|
||||
"patent_country",
|
||||
"patent_number",
|
||||
"standard_number",
|
||||
"report_number",
|
||||
"arxiv_id",
|
||||
"keywords",
|
||||
"citation_key",
|
||||
)
|
||||
@classmethod
|
||||
def _blank_to_none(cls, value: str | None) -> str | None:
|
||||
"""Treat a whitespace-only string as "not set" rather than as a value."""
|
||||
if value is None:
|
||||
return None
|
||||
return value.strip() or None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _check_online_resource(self) -> "ReferenceBase":
|
||||
"""An online resource must say where it is.
|
||||
|
||||
GB/T 7714 prints a URL for ``[EB/OL]`` and nothing else identifies the
|
||||
document; a row typed as a web page or a dataset with no URL is a row
|
||||
that cannot be cited.
|
||||
"""
|
||||
if self.type in ("web-page", "dataset") and not (self.url or self.doi):
|
||||
raise ValueError("网页/数据集类型的文献必须至少给出 URL 或 DOI")
|
||||
return self
|
||||
|
||||
|
||||
class ReferenceCreate(ReferenceBase):
|
||||
"""Payload for ``POST /references``."""
|
||||
|
||||
|
||||
class ReferenceUpdate(BaseModel):
|
||||
"""Payload for ``PATCH /references/{id}`` — every part optional.
|
||||
|
||||
``authors`` is a full replacement when present, like a template's
|
||||
placements: the citation prints the list in the order it arrives.
|
||||
"""
|
||||
|
||||
type: str | None = Field(default=None, max_length=32)
|
||||
title: str | None = Field(default=None, min_length=1, max_length=512)
|
||||
subtitle: str | None = Field(default=None, max_length=512)
|
||||
title_translated: str | None = Field(default=None, max_length=512)
|
||||
title_short: str | None = Field(default=None, max_length=255)
|
||||
container_title: str | None = Field(default=None, max_length=512)
|
||||
container_title_short: str | None = Field(default=None, max_length=255)
|
||||
series_title: str | None = Field(default=None, max_length=512)
|
||||
series_number: str | None = Field(default=None, max_length=32)
|
||||
edition: str | None = Field(default=None, max_length=64)
|
||||
volume: str | None = Field(default=None, max_length=32)
|
||||
issue: str | None = Field(default=None, max_length=32)
|
||||
pages: str | None = Field(default=None, max_length=64)
|
||||
article_number: str | None = Field(default=None, max_length=64)
|
||||
publisher: str | None = Field(default=None, max_length=255)
|
||||
publisher_place: str | None = Field(default=None, max_length=255)
|
||||
event_title: str | None = Field(default=None, max_length=512)
|
||||
event_place: str | None = Field(default=None, max_length=255)
|
||||
event_date: str | None = Field(default=None, max_length=64)
|
||||
publication_year: int | None = Field(default=None, ge=1, le=2999)
|
||||
publication_month: int | None = Field(default=None, ge=1, le=12)
|
||||
publication_day: int | None = Field(default=None, ge=1, le=31)
|
||||
accessed_date: date | None = None
|
||||
doi: str | None = Field(default=None, max_length=255)
|
||||
isbn: str | None = Field(default=None, max_length=32)
|
||||
issn: str | None = Field(default=None, max_length=32)
|
||||
url: str | None = Field(default=None, max_length=1024)
|
||||
patent_country: str | None = Field(default=None, max_length=64)
|
||||
patent_number: str | None = Field(default=None, max_length=64)
|
||||
standard_number: str | None = Field(default=None, max_length=64)
|
||||
report_number: str | None = Field(default=None, max_length=64)
|
||||
arxiv_id: str | None = Field(default=None, max_length=64)
|
||||
abstract: str | None = None
|
||||
keywords: str | None = Field(default=None, max_length=512)
|
||||
note: str | None = None
|
||||
citation_key: str | None = Field(default=None, max_length=64)
|
||||
language: str | None = Field(default=None, max_length=16)
|
||||
authors: list[ReferenceAuthorInput] | None = None
|
||||
|
||||
@field_validator("title")
|
||||
@classmethod
|
||||
def _require_title(cls, value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
text = value.strip()
|
||||
if not text:
|
||||
raise ValueError("文献题名不能为空")
|
||||
return text
|
||||
|
||||
@field_validator("type")
|
||||
@classmethod
|
||||
def _check_type(cls, value: str | None) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
if value not in REFERENCE_TYPES:
|
||||
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
|
||||
return value
|
||||
|
||||
@field_validator("language")
|
||||
@classmethod
|
||||
def _check_language(cls, value: str | None) -> str | None:
|
||||
if value is None or value == "":
|
||||
return None
|
||||
if value not in REFERENCE_LANGUAGES:
|
||||
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
|
||||
return value
|
||||
|
||||
@field_validator("doi")
|
||||
@classmethod
|
||||
def _normalise_doi(cls, value: str | None) -> str | None:
|
||||
return normalize_doi(value)
|
||||
|
||||
|
||||
class ReferenceListItem(BaseModel):
|
||||
"""A reference as it appears in the library table."""
|
||||
|
||||
id: int
|
||||
type: str
|
||||
title: str
|
||||
subtitle: str | None
|
||||
container_title: str | None
|
||||
publication_year: int | None
|
||||
volume: str | None
|
||||
issue: str | None
|
||||
pages: str | None
|
||||
doi: str | None
|
||||
#: The printed author list, joined — a table row needs the line, not the
|
||||
#: rows behind it.
|
||||
authors_line: str
|
||||
author_count: int
|
||||
#: How many sentences cite it. Drives the refusal message on delete, and
|
||||
#: tells the writer at a glance what is already used.
|
||||
citation_count: int
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class ReferenceRead(BaseModel):
|
||||
"""A reference with everything the table does not need."""
|
||||
|
||||
id: int
|
||||
type: str
|
||||
title: str
|
||||
subtitle: str | None
|
||||
title_translated: str | None
|
||||
title_short: str | None
|
||||
container_title: str | None
|
||||
container_title_short: str | None
|
||||
series_title: str | None
|
||||
series_number: str | None
|
||||
edition: str | None
|
||||
volume: str | None
|
||||
issue: str | None
|
||||
pages: str | None
|
||||
article_number: str | None
|
||||
publisher: str | None
|
||||
publisher_place: str | None
|
||||
event_title: str | None
|
||||
event_place: str | None
|
||||
event_date: str | None
|
||||
publication_year: int | None
|
||||
publication_month: int | None
|
||||
publication_day: int | None
|
||||
accessed_date: date | None
|
||||
doi: str | None
|
||||
isbn: str | None
|
||||
issn: str | None
|
||||
url: str | None
|
||||
patent_country: str | None
|
||||
patent_number: str | None
|
||||
standard_number: str | None
|
||||
report_number: str | None
|
||||
arxiv_id: str | None
|
||||
abstract: str | None
|
||||
keywords: str | None
|
||||
note: str | None
|
||||
citation_key: str | None
|
||||
language: str | None
|
||||
authors: list[ReferenceAuthorRead]
|
||||
citation_count: int
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class ReferenceCitationRead(BaseModel):
|
||||
"""One sentence that cites a reference — the reverse lookup.
|
||||
|
||||
"Which of my sentences use this article?" is the question a reference
|
||||
library exists to answer, and it needs the sentence, the paragraph it sits
|
||||
in, and the paper it belongs to.
|
||||
"""
|
||||
|
||||
citation_id: int
|
||||
sentence_id: int
|
||||
sentence_content: str
|
||||
quote: str
|
||||
paper_id: int
|
||||
paper_title: str
|
||||
paper_template_filed_sort: int
|
||||
paragraph_name: str | None = None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ReferenceAuthorInput",
|
||||
"ReferenceAuthorRead",
|
||||
"ReferenceBase",
|
||||
"ReferenceCitationRead",
|
||||
"ReferenceCreate",
|
||||
"ReferenceListItem",
|
||||
"ReferenceRead",
|
||||
"ReferenceUpdate",
|
||||
"normalize_doi",
|
||||
]
|
||||
Reference in New Issue
Block a user