backend: add the reference library (参考文献)
One table per document the writer has read, and one for its contributors:
reference id, type, title, subtitle, title_translated,
container_title, volume, issue, pages, publisher,
publisher_place, event_*, publication_year/month/day,
accessed_date, doi, isbn, issn, url, patent_*,
standard_number, report_number, arxiv_id, abstract,
keywords, note, citation_key, language
reference_author reference_id, sort, name, family_name, given_name, role
The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:
* **Partial dates.** Year, month and day are separate nullable integers, which
is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
"S. Zhang"), so the printed form and its two halves are stored separately —
display reads `name`, a reformatting style reads the halves, and an
institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
`S1` and `e12345`, and a citation reproduces what is printed.
The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.
Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.
API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.
Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
This commit is contained in:
+79
-3
@@ -58,8 +58,8 @@ Browser ──HTTP/JSON──▶ FastAPI ──SQLAlchemy──▶ TiDB (k3s)
|
||||
|
||||
## Domain Model
|
||||
|
||||
Six tables. Every one is prefixed with the module it belongs to, and a model
|
||||
class is named after its table, so the schema can be read as three groups:
|
||||
Nine tables. Every one is prefixed with the module it belongs to, and a model
|
||||
class is named after its table, so the schema can be read as four groups:
|
||||
|
||||
```
|
||||
template_title one reusable heading (字段库)
|
||||
@@ -79,7 +79,14 @@ paper_sentence one sentence, at one position, in one paper
|
||||
id, paper_id → paper, template_id, paper_template_filed_sort, sort, content
|
||||
|
||||
paper_sentence_reference the citations of one sentence
|
||||
id, sentence_id → paper_sentence, reference_id, quote, sort
|
||||
id, sentence_id → paper_sentence, reference_id → reference, quote, sort
|
||||
|
||||
reference the 参考文献 module: the literature itself
|
||||
id, type, title, subtitle, container_title, volume, issue, pages,
|
||||
publisher, publisher_place, publication_year, doi, url, …
|
||||
|
||||
reference_author its contributors, in print order
|
||||
id, reference_id → reference, sort, name, family_name, given_name, role
|
||||
```
|
||||
|
||||
### How the tables are named
|
||||
@@ -194,6 +201,49 @@ written against. It is kept in step with the paper's current template and is
|
||||
provenance rather than a lookup key: rendering never filters on it, which is
|
||||
precisely why content survives a switch.
|
||||
|
||||
### The reference library holds literature, not citations
|
||||
|
||||
``reference`` is the *literature*: one row per document the writer has read, and
|
||||
it is **global** — the same article is cited from several papers, so the entry
|
||||
cannot belong to one of them. The *citation* is a different fact — "this
|
||||
sentence quotes this passage" — and it lives on the paper side, in
|
||||
``paper_sentence_reference``, which carries the quote and the citation's order
|
||||
inside the sentence.
|
||||
|
||||
So the relation is many-to-many in both directions, through that table:
|
||||
|
||||
```
|
||||
paper_sentence ──< paper_sentence_reference >── reference
|
||||
(quote, sort) (title, year, …)
|
||||
```
|
||||
|
||||
Two questions follow from it, and each is answered from its own side:
|
||||
"what does this sentence quote?" is a lookup by ``sentence_id``, and "where have
|
||||
I used this article?" is a lookup by ``reference_id``
|
||||
(``GET /references/{id}/citations``). One column on ``paper_sentence`` could not
|
||||
express the first of those — it would cap a sentence at one citation — which is
|
||||
why the relation is a table rather than a foreign key.
|
||||
|
||||
#### Why these columns and no others
|
||||
|
||||
The field set is the intersection of the formats a paper has to survive:
|
||||
|
||||
| Format | What it decided |
|
||||
|---|---|
|
||||
| CSL JSON (``csl-data.json``: 103 fields, 45 types) | the field names, and partial dates as year / month / day rather than one ``DATE`` |
|
||||
| GB/T 7714-2015 | every entry carries a ``type`` with its marker; patents need ``patent_country`` + ``patent_number``, standards ``standard_number``, online resources ``accessed_date`` |
|
||||
| BibTeX / RIS | ``citation_key``, and the one-field-per-fact layout that makes an export a mapping rather than a parse |
|
||||
|
||||
Contributors are a child table, not a column, and each row stores the printed
|
||||
name *and* its two halves. That is not duplication: a citation prints 「张三」,
|
||||
``Zhang, S.`` or ``S. Zhang`` depending on the style, so the printed form is the
|
||||
authority for display and the halves are the authority for rewriting a name —
|
||||
and an institutional author (「世界卫生组织」) has no halves at all.
|
||||
|
||||
The 103 CSL fields include roles this project will never print (narrator,
|
||||
performer, curator) and archival fields (call-number, archive-place). They are
|
||||
left out on purpose; see :mod:`app.models.reference`.
|
||||
|
||||
### A sentence is a line, not parsed text
|
||||
|
||||
Nothing in the writing path splits text. There is no sentence detector, no
|
||||
@@ -299,6 +349,11 @@ All routes are mounted under `/api`. Interactive docs at `/docs`.
|
||||
| `POST` | `/papers` | title + template + author/status/keywords/journal; no sentence rows are created |
|
||||
| `GET` `PATCH` `DELETE` | `/papers/{id}` | `PATCH` with `template_id` **is** the template switch |
|
||||
| `POST` | `/papers/batch-delete` | body `{ "ids": [...] }` |
|
||||
| `GET` | `/references` | `keyword` (题名/刊名/作者/DOI/关键词/备注), `type`, `year`, `cited` |
|
||||
| `POST` | `/references` | the whole entry, `authors` in print order |
|
||||
| `GET` `PATCH` `DELETE` | `/references/{id}` | `PATCH` with `authors` replaces the list |
|
||||
| `POST` | `/references/batch-delete` | body `{ "ids": [...] }` |
|
||||
| `GET` | `/references/{id}/citations` | which sentences cite it, and in which paragraph |
|
||||
| `GET` | `/papers/{id}/document` | the whole paper: paragraphs in order, citations, warnings |
|
||||
| `GET` | `/papers/{id}/paragraphs/{sort}` | one paragraph, assembled as the document renders it |
|
||||
| `PUT` | `/papers/{id}/paragraphs/{sort}` | full replacement; optional `target_sort` moves it |
|
||||
@@ -327,6 +382,14 @@ Conventions worth knowing:
|
||||
- `keywords` is stored as a canonical ``, ``-joined string; `,`、`;` and `;`
|
||||
are accepted on write and de-duplicated.
|
||||
- Paper `status` is one of `draft`, `writing`, `done`.
|
||||
- A citation's ``reference_id`` is checked against the library: citing
|
||||
literature that does not exist is a `400` naming the ids, not a foreign-key
|
||||
error from the driver. A citation with no ``reference_id`` stays legal — a
|
||||
quote may be written before it is linked.
|
||||
- Deleting a reference a sentence cites is refused with `409`, naming the entry
|
||||
and how many citations use it.
|
||||
- A DOI is stored without its ``https://doi.org/`` prefix, so the column holds
|
||||
one spelling whichever way it was pasted.
|
||||
|
||||
## MCP server
|
||||
|
||||
@@ -513,6 +576,7 @@ paper-doc/
|
||||
│ ├── alembic/ # migration environment and revisions
|
||||
│ ├── scripts/seed.py # idempotent seed for the title library + templates
|
||||
│ ├── scripts/smoke_papers.py # end-to-end check of the writing loop
|
||||
│ ├── scripts/smoke_references.py # end-to-end check of the reference library
|
||||
│ ├── scripts/mcp_server.py # MCP entry point (stdio / http)
|
||||
│ ├── scripts/smoke_mcp.py # the same loop, spoken over MCP by a real client
|
||||
│ ├── alembic.ini
|
||||
@@ -592,6 +656,18 @@ cd backend
|
||||
.venv/bin/python scripts/seed.py --reset # empty the tables first
|
||||
```
|
||||
|
||||
**Checking the reference library end to end**
|
||||
|
||||
`scripts/smoke_references.py` adds literature, finds it again by author, type,
|
||||
year and whether it has been cited, edits it, cites it from a sentence, looks
|
||||
the citation up from the reference side, and checks that a cited entry cannot be
|
||||
deleted. It cleans up after itself:
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
.venv/bin/python scripts/smoke_references.py
|
||||
```
|
||||
|
||||
**Checking the paper feature end to end**
|
||||
|
||||
`scripts/smoke_papers.py` walks the whole writing loop against a running API —
|
||||
|
||||
Reference in New Issue
Block a user