Files
paper-doc/backend/app/models/reference.py
T
govin d7bb777264 backend: add the reference library (参考文献)
One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
2026-09-22 17:13:32 +08:00

243 lines
12 KiB
Python

"""The reference library (参考文献库) — the literature a paper cites.
One row is one document the writer has read and may cite: a journal article, a
conference paper, a book, a patent, a standard, a dataset. It is **global**, not
owned by a paper, because the same article is cited from several papers — and
the citation itself (which sentence quotes which passage) belongs to the paper,
in :class:`~app.models.paper_sentence_reference.PaperSentenceReference`.
Why these columns
-----------------
The field set is the intersection of the formats a Chinese journal paper
actually has to survive, so that one row can be rendered as any of them without
a second home for anything:
* **CSL JSON** (``csl-data.json``, 103 fields / 45 types) — what Zotero,
Mendeley and Pandoc exchange. Our names follow it wherever a field is the
same field: ``title``, ``container-title``, ``volume``, ``issue``, ``page``,
``publisher``, ``publisher-place``, ``edition``, ``DOI``, ``ISBN``,
``ISSN``, ``URL``, ``abstract``, ``keyword``, ``language``, ``note``,
``citation-key``, ``event-title``, ``event-place``, ``issued``, ``accessed``.
* **GB/T 7714-2015** — the Chinese national standard a domestic journal will
demand. It is why every entry carries a ``type`` with a marker, and why
patents need ``patent_country`` and ``patent_number``, standards need
``standard_number``, and online resources need ``accessed_date``.
* **BibTeX / RIS** — the export formats a reference manager is judged by.
``citation_key`` is the BibTeX key; the rest map one-to-one with the fields
above.
Contributors are **not** columns: a document has an ordered list of them, each
with a role, which is what :class:`~app.models.reference_author.ReferenceAuthor`
holds. Formats disagree about names anyway — 「张三」, ``Zhang San``,
``Zhang, S.`` are the same person in three styles — so the printed form and the
two halves are stored separately and a style picks what it needs.
What is deliberately missing
----------------------------
The 103 CSL fields include roles this project will never print (narrator,
performer, chair, illustrator, curator…) and archival fields (call-number,
archive-place). Adding a column for each would be a schema nobody reads. They
can arrive when a real citation needs one.
"""
from datetime import date
from typing import TYPE_CHECKING
from sqlalchemy import Date, Index, Integer, SmallInteger, String, Text, text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.db.base import Base
from app.models.mixins import TimestampMixin
if TYPE_CHECKING: # pragma: no cover - typing only
from app.models.reference_author import ReferenceAuthor
#: Document types this library holds, with the marker each format uses for it.
#:
#: ====================== ================ ========= ==========================
#: our value CSL type GB/T [X] example
#: ====================== ================ ========= ==========================
#: journal-article article-journal [J] 期刊论文
#: book book [M] 专著
#: book-chapter chapter [M] 析出文献(书里的一章)
#: conference-paper paper-conference [C] 会议论文
#: thesis thesis [D] 学位论文
#: patent patent [P] 专利
#: standard standard [S] 标准
#: report report [R] 报告
#: newspaper-article article-newspaper[N] 报纸文章
#: web-page webpage [EB/OL] 网页/在线资源
#: dataset dataset [DS] 数据集
#: software software [CP] 计算机程序
#: preprint article [EB/OL] 预印本(arXiv 等)
#: other document [Z] 其他
#: ====================== ================ ========= ==========================
REFERENCE_TYPES: tuple[str, ...] = (
"journal-article",
"book",
"book-chapter",
"conference-paper",
"thesis",
"patent",
"standard",
"report",
"newspaper-article",
"web-page",
"dataset",
"software",
"preprint",
"other",
)
#: The roles a contributor can have. ``author`` is the default and the one that
#: prints in the citation; the others carry the 编 / 译 / 主编 marks that GB/T
#: 7714 puts after a name.
REFERENCE_AUTHOR_ROLES: tuple[str, ...] = (
"author",
"editor",
"translator",
"compiler",
)
#: Languages a row is written in — used to choose which title is primary and
#: which is the 并列题名.
REFERENCE_LANGUAGES: tuple[str, ...] = ("zh", "en", "ja", "other")
class Reference(TimestampMixin, Base):
"""One document in the reference library."""
__tablename__ = "reference"
__table_args__ = (
# The list endpoint filters by type and orders by year, which this
# serves. There is deliberately no index on ``title``: the search box
# matches with a leading wildcard, which no B-tree can use.
Index("ix_reference_type_year", "type", "publication_year"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
#: One of :data:`REFERENCE_TYPES`. Stored as the token, never the Chinese
#: label, so the wording can change without a migration.
type: Mapped[str] = mapped_column(
String(32),
nullable=False,
default="journal-article",
server_default=text("'journal-article'"),
)
# --- the document's name ------------------------------------------------
#: 题名, as printed.
title: Mapped[str] = mapped_column(String(512), nullable=False)
#: 其他题名信息 — the part after a colon (": 基于……的实证研究").
subtitle: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: 并列题名 / 译名 — the title in the other language, which a bilingual
#: manuscript prints alongside the first.
title_translated: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: Short form for in-text citations (CSL ``title-short``).
title_short: Mapped[str | None] = mapped_column(String(255), nullable=True)
# --- where it was published --------------------------------------------
#: 刊名 / 书名 / 论文集名 / 报纸名 — the container the entry sits in.
container_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: 刊名缩写 (CSL ``container-title-short``). Some styles print only this.
container_title_short: Mapped[str | None] = mapped_column(
String(255), nullable=True
)
#: 丛书 / 系列名, and its number.
series_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
series_number: Mapped[str | None] = mapped_column(String(32), nullable=True)
#: 版本项 — "3版", "2nd ed.".
edition: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 卷 and 期. Strings rather than integers: journals print "12(Suppl.)",
#: "S1" and "春季号", and a citation has to reproduce what is printed.
volume: Mapped[str | None] = mapped_column(String(32), nullable=True)
issue: Mapped[str | None] = mapped_column(String(32), nullable=True)
#: 页码, as printed — "1024-1032", "e12345", "12-15, 20".
pages: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 文章编号 / eLocation, for journals that have no page numbers.
article_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- publisher ----------------------------------------------------------
#: 出版者. For a thesis this is the 保存者 (the university), for a report the
#: issuing body, for a standard the publisher — the same role in three
#: formats, which is why there is one column and not three.
publisher: Mapped[str | None] = mapped_column(String(255), nullable=True)
#: 出版地.
publisher_place: Mapped[str | None] = mapped_column(String(255), nullable=True)
# --- events (conferences) ----------------------------------------------
#: 会议名称 / 会议地点 / 会期. GB/T prints the proceedings title, APA and
#: IEEE print the meeting, and a conference paper usually needs both.
event_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
event_place: Mapped[str | None] = mapped_column(String(255), nullable=True)
#: Free text on purpose: conferences run "2024-05-01/03" and styles print
#: the range as written.
event_date: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- dates --------------------------------------------------------------
# Partial dates, exactly as CSL models ``issued``: a book has a year, a
# newspaper article has a day, a patent has its 公告日期.
publication_year: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
publication_month: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
publication_day: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
#: 引用日期 — required by GB/T 7714 for anything online, because a URL's
#: content can change after it was read.
accessed_date: Mapped[date | None] = mapped_column(Date, nullable=True)
# --- identifiers --------------------------------------------------------
#: Canonical DOI, without the ``https://doi.org/`` prefix.
doi: Mapped[str | None] = mapped_column(String(255), nullable=True)
isbn: Mapped[str | None] = mapped_column(String(32), nullable=True)
issn: Mapped[str | None] = mapped_column(String(32), nullable=True)
url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
#: 专利国别 + 专利号 — GB/T 7714 prints both inside the [P] marker.
patent_country: Mapped[str | None] = mapped_column(String(64), nullable=True)
patent_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 标准编号 ("GB/T 7714-2015") and 报告编号.
standard_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
report_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: Preprint identifier ("arXiv:2405.12345"), which is how an AI paper is
#: cited before it has a DOI.
arxiv_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- the writer's own notes --------------------------------------------
abstract: Mapped[str | None] = mapped_column(Text, nullable=True)
#: 文献关键词, comma-separated like the paper's own keywords.
keywords: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: Free text: why it was kept, what it is for, where the PDF is.
note: Mapped[str | None] = mapped_column(Text, nullable=True)
# --- library metadata ---------------------------------------------------
#: The BibTeX key (CSL ``citation-key``), for export.
citation_key: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: One of :data:`REFERENCE_LANGUAGES`; decides which title leads.
language: Mapped[str | None] = mapped_column(String(16), nullable=True)
#: The contributors, in the order the citation prints them.
authors: Mapped[list["ReferenceAuthor"]] = relationship(
back_populates="reference",
cascade="all, delete-orphan",
order_by="ReferenceAuthor.sort, ReferenceAuthor.id",
lazy="selectin",
)
def __repr__(self) -> str: # pragma: no cover - debugging aid
return f"<Reference id={self.id} type={self.type} title={self.title!r}>"