backend: add the reference library (参考文献)

One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
This commit is contained in:
2026-09-22 17:13:32 +08:00
parent 103b5544ba
commit d7bb777264
18 changed files with 1966 additions and 16 deletions
@@ -0,0 +1,162 @@
"""create the reference library (参考文献)
Two tables for the literature itself, and one foreign key that has been waiting
for them:
``reference`` one document: an article, a book, a patent, a dataset.
Global rather than owned by a paper, because the same
article is cited from several papers.
``reference_author`` its contributors, in the order the citation prints them.
A child table rather than a column: a citation prints an
ordered list with 编 / 译 marks, and styles rewrite names
(「张三」 / "Zhang, S." / "S. Zhang"), so the printed form
and its two halves are stored separately.
The columns are the intersection of CSL JSON, GB/T 7714-2015, BibTeX and RIS —
the reasoning is on :mod:`app.models.reference`. Two notes that belong to the
schema rather than the model:
* Partial dates are three nullable integers (year, month, day) rather than one
``DATE``, because that is what CSL models as ``issued``: a book has a year, a
newspaper article a day, a patent its 公告日期, and a citation must not invent
the rest.
* ``volume`` / ``issue`` / ``pages`` are strings. Journals print ``12(Suppl.)``,
``S1`` and ``e12345``, and a citation reproduces what is printed.
``paper_sentence_reference.reference_id`` finally becomes a real foreign key,
``ON DELETE RESTRICT``: a citation must not be able to point at literature that
has been removed from the library. The library is empty at this revision and so
is the citation table, so no existing row can violate it.
Revision ID: e4b7c1f9a036
Revises: b19e4d7a3c58
Create Date: 2026-09-22
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "e4b7c1f9a036"
down_revision: str | None = "b19e4d7a3c58"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_NOW = sa.text("CURRENT_TIMESTAMP")
#: (name, type) for every text column, in one place so the model and the table
#: cannot drift apart on a length.
_TEXT_COLUMNS: tuple[tuple[str, int], ...] = (
("subtitle", 512),
("title_translated", 512),
("title_short", 255),
("container_title", 512),
("container_title_short", 255),
("series_title", 512),
("series_number", 32),
("edition", 64),
("volume", 32),
("issue", 32),
("pages", 64),
("article_number", 64),
("publisher", 255),
("publisher_place", 255),
("event_title", 512),
("event_place", 255),
("event_date", 64),
("doi", 255),
("isbn", 32),
("issn", 32),
("url", 1024),
("patent_country", 64),
("patent_number", 64),
("standard_number", 64),
("report_number", 64),
("arxiv_id", 64),
("keywords", 512),
("citation_key", 64),
("language", 16),
)
def upgrade() -> None:
op.create_table(
"reference",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column(
"type",
sa.String(length=32),
server_default=sa.text("'journal-article'"),
nullable=False,
),
sa.Column("title", sa.String(length=512), nullable=False),
*[
sa.Column(name, sa.String(length=length), nullable=True)
for name, length in _TEXT_COLUMNS
],
sa.Column("publication_year", sa.SmallInteger(), nullable=True),
sa.Column("publication_month", sa.SmallInteger(), nullable=True),
sa.Column("publication_day", sa.SmallInteger(), nullable=True),
sa.Column("accessed_date", sa.Date(), nullable=True),
sa.Column("abstract", sa.Text(), nullable=True),
sa.Column("note", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(), server_default=_NOW, nullable=False),
sa.Column("updated_at", sa.DateTime(), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"ix_reference_type_year",
"reference",
["type", "publication_year"],
unique=False,
)
op.create_table(
"reference_author",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("reference_id", sa.Integer(), nullable=False),
sa.Column("sort", sa.Integer(), nullable=False),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column("family_name", sa.String(length=128), nullable=True),
sa.Column("given_name", sa.String(length=128), nullable=True),
sa.Column(
"role",
sa.String(length=16),
server_default=sa.text("'author'"),
nullable=False,
),
sa.ForeignKeyConstraint(["reference_id"], ["reference.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"ix_reference_author_reference_sort",
"reference_author",
["reference_id", "sort"],
unique=False,
)
# The citation column stops being a loose integer: it now points at the
# library, and the database refuses a citation to literature that is gone.
op.create_foreign_key(
"fk_paper_sentence_reference_reference",
"paper_sentence_reference",
"reference",
["reference_id"],
["id"],
ondelete="RESTRICT",
)
def downgrade() -> None:
op.drop_constraint(
"fk_paper_sentence_reference_reference",
"paper_sentence_reference",
type_="foreignkey",
)
op.drop_index("ix_reference_author_reference_sort", table_name="reference_author")
op.drop_table("reference_author")
op.drop_index("ix_reference_type_year", table_name="reference")
op.drop_table("reference")