backend: add the reference library (参考文献)

One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
This commit is contained in:
2026-09-22 17:13:32 +08:00
parent 103b5544ba
commit d7bb777264
18 changed files with 1966 additions and 16 deletions
@@ -0,0 +1,162 @@
"""create the reference library (参考文献)
Two tables for the literature itself, and one foreign key that has been waiting
for them:
``reference`` one document: an article, a book, a patent, a dataset.
Global rather than owned by a paper, because the same
article is cited from several papers.
``reference_author`` its contributors, in the order the citation prints them.
A child table rather than a column: a citation prints an
ordered list with 编 / 译 marks, and styles rewrite names
(「张三」 / "Zhang, S." / "S. Zhang"), so the printed form
and its two halves are stored separately.
The columns are the intersection of CSL JSON, GB/T 7714-2015, BibTeX and RIS —
the reasoning is on :mod:`app.models.reference`. Two notes that belong to the
schema rather than the model:
* Partial dates are three nullable integers (year, month, day) rather than one
``DATE``, because that is what CSL models as ``issued``: a book has a year, a
newspaper article a day, a patent its 公告日期, and a citation must not invent
the rest.
* ``volume`` / ``issue`` / ``pages`` are strings. Journals print ``12(Suppl.)``,
``S1`` and ``e12345``, and a citation reproduces what is printed.
``paper_sentence_reference.reference_id`` finally becomes a real foreign key,
``ON DELETE RESTRICT``: a citation must not be able to point at literature that
has been removed from the library. The library is empty at this revision and so
is the citation table, so no existing row can violate it.
Revision ID: e4b7c1f9a036
Revises: b19e4d7a3c58
Create Date: 2026-09-22
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "e4b7c1f9a036"
down_revision: str | None = "b19e4d7a3c58"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_NOW = sa.text("CURRENT_TIMESTAMP")
#: (name, type) for every text column, in one place so the model and the table
#: cannot drift apart on a length.
_TEXT_COLUMNS: tuple[tuple[str, int], ...] = (
("subtitle", 512),
("title_translated", 512),
("title_short", 255),
("container_title", 512),
("container_title_short", 255),
("series_title", 512),
("series_number", 32),
("edition", 64),
("volume", 32),
("issue", 32),
("pages", 64),
("article_number", 64),
("publisher", 255),
("publisher_place", 255),
("event_title", 512),
("event_place", 255),
("event_date", 64),
("doi", 255),
("isbn", 32),
("issn", 32),
("url", 1024),
("patent_country", 64),
("patent_number", 64),
("standard_number", 64),
("report_number", 64),
("arxiv_id", 64),
("keywords", 512),
("citation_key", 64),
("language", 16),
)
def upgrade() -> None:
op.create_table(
"reference",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column(
"type",
sa.String(length=32),
server_default=sa.text("'journal-article'"),
nullable=False,
),
sa.Column("title", sa.String(length=512), nullable=False),
*[
sa.Column(name, sa.String(length=length), nullable=True)
for name, length in _TEXT_COLUMNS
],
sa.Column("publication_year", sa.SmallInteger(), nullable=True),
sa.Column("publication_month", sa.SmallInteger(), nullable=True),
sa.Column("publication_day", sa.SmallInteger(), nullable=True),
sa.Column("accessed_date", sa.Date(), nullable=True),
sa.Column("abstract", sa.Text(), nullable=True),
sa.Column("note", sa.Text(), nullable=True),
sa.Column("created_at", sa.DateTime(), server_default=_NOW, nullable=False),
sa.Column("updated_at", sa.DateTime(), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"ix_reference_type_year",
"reference",
["type", "publication_year"],
unique=False,
)
op.create_table(
"reference_author",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("reference_id", sa.Integer(), nullable=False),
sa.Column("sort", sa.Integer(), nullable=False),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column("family_name", sa.String(length=128), nullable=True),
sa.Column("given_name", sa.String(length=128), nullable=True),
sa.Column(
"role",
sa.String(length=16),
server_default=sa.text("'author'"),
nullable=False,
),
sa.ForeignKeyConstraint(["reference_id"], ["reference.id"], ondelete="CASCADE"),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"ix_reference_author_reference_sort",
"reference_author",
["reference_id", "sort"],
unique=False,
)
# The citation column stops being a loose integer: it now points at the
# library, and the database refuses a citation to literature that is gone.
op.create_foreign_key(
"fk_paper_sentence_reference_reference",
"paper_sentence_reference",
"reference",
["reference_id"],
["id"],
ondelete="RESTRICT",
)
def downgrade() -> None:
op.drop_constraint(
"fk_paper_sentence_reference_reference",
"paper_sentence_reference",
type_="foreignkey",
)
op.drop_index("ix_reference_author_reference_sort", table_name="reference_author")
op.drop_table("reference_author")
op.drop_index("ix_reference_type_year", table_name="reference")
op.drop_table("reference")
+2 -1
View File
@@ -2,10 +2,11 @@
from fastapi import APIRouter from fastapi import APIRouter
from app.api.routes import health, papers, template_titles, templates from app.api.routes import health, papers, references, template_titles, templates
api_router = APIRouter() api_router = APIRouter()
api_router.include_router(health.router) api_router.include_router(health.router)
api_router.include_router(papers.router) api_router.include_router(papers.router)
api_router.include_router(references.router)
api_router.include_router(template_titles.router) api_router.include_router(template_titles.router)
api_router.include_router(templates.router) api_router.include_router(templates.router)
+1
View File
@@ -246,6 +246,7 @@ def replace_paragraph(
stale copy. stale copy.
""" """
paper = _get_or_404(db, paper_id) paper = _get_or_404(db, paper_id)
crud.replace_paragraph(db, paper, field_sort, payload) crud.replace_paragraph(db, paper, field_sort, payload)
return crud.build_document(db, paper) return crud.build_document(db, paper)
+178
View File
@@ -0,0 +1,178 @@
"""Reference-library endpoints (参考文献).
The library is global and reusable: an article is added once and then cited
from any number of sentences in any number of papers. That is why deleting one
is guarded — the citation rows are the only thing tying a paper's 参考文献 list
to the literature, and dropping a cited reference would quietly empty those
citations rather than tidy up the library.
The reverse lookup lives here too: ``GET /references/{id}/citations`` answers
"which of my sentences use this?", which is the question a library exists to
answer once the writing has started.
"""
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from app.crud import reference as crud
from app.db.session import get_db
from app.models import REFERENCE_TYPES, Reference
from app.schemas.common import BatchDeleteRequest, BatchDeleteResult, PageResult
from app.schemas.reference import (
ReferenceCitationRead,
ReferenceCreate,
ReferenceListItem,
ReferenceRead,
ReferenceUpdate,
)
router = APIRouter(prefix="/references", tags=["references"])
def _get_or_404(db: Session, reference_id: int) -> Reference:
reference = crud.get(db, reference_id)
if reference is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"参考文献 {reference_id} 不存在",
)
return reference
def _assert_not_cited(db: Session, references: list[Reference]) -> None:
"""Refuse the delete while any sentence cites the reference.
All offenders are reported at once rather than one per attempt, so a batch
delete does not turn into trial and error. The message says what to do about
it, because "in use" without a next step is a dead end.
"""
counts = crud.citation_counts(db, [reference.id for reference in references])
if not counts:
return
titles = {reference.id: reference.title for reference in references}
blockers = "、".join(
f"“{titles.get(reference_id, reference_id)}”({count} 处引用)"
for reference_id, count in sorted(counts.items())
)
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=(
"以下文献正被论文引用,请先在段落里移除对应引用:"
+ blockers
),
)
@router.get("", response_model=PageResult[ReferenceListItem], summary="List references")
def list_references(
db: Session = Depends(get_db),
keyword: str | None = Query(
default=None, description="按题名、刊名、作者、DOI、关键词或备注模糊搜索"
),
type_filter: str | None = Query(
default=None,
alias="type",
description="按文献类型过滤:" + " / ".join(REFERENCE_TYPES),
),
year: int | None = Query(default=None, ge=1, le=2999, description="按出版年过滤"),
cited: bool | None = Query(
default=None, description="true 只看已被引用的,false 只看还没用过的"
),
page: int = Query(default=1, ge=1),
page_size: int = Query(default=20, ge=1, le=200),
) -> PageResult[ReferenceListItem]:
"""Browse the library, newest publication first."""
items, total = crud.list_references(
db,
keyword=keyword,
type_=type_filter,
year=year,
cited_only=cited,
page=page,
page_size=page_size,
)
return PageResult.build(items=items, total=total, page=page, page_size=page_size)
@router.post(
"",
response_model=ReferenceRead,
status_code=status.HTTP_201_CREATED,
summary="Add a reference",
)
def create_reference(
payload: ReferenceCreate,
db: Session = Depends(get_db),
) -> ReferenceRead:
"""Add one document to the library, with its contributors in order."""
return crud.read(db, crud.create(db, payload))
@router.post(
"/batch-delete",
response_model=BatchDeleteResult,
summary="Delete several references",
)
def batch_delete_references(
payload: BatchDeleteRequest,
db: Session = Depends(get_db),
) -> BatchDeleteResult:
"""Delete the given references, refusing wholesale if any is cited."""
references = crud.get_many(db, payload.ids)
_assert_not_cited(db, references)
return BatchDeleteResult(deleted=crud.delete_many(db, payload.ids))
@router.get("/{reference_id}", response_model=ReferenceRead, summary="Fetch one reference")
def get_reference(reference_id: int, db: Session = Depends(get_db)) -> ReferenceRead:
"""Return one reference, with its contributors and citation count."""
return crud.read(db, _get_or_404(db, reference_id))
@router.get(
"/{reference_id}/citations",
response_model=list[ReferenceCitationRead],
summary="Which sentences cite this reference",
)
def list_reference_citations(
reference_id: int, db: Session = Depends(get_db)
) -> list[ReferenceCitationRead]:
"""Return every sentence that cites this reference, with its paragraph.
This is the direction the paper side cannot answer: a sentence knows which
articles it quotes, but only the library can say where an article has been
used — which is what tells the writer whether it is safe to delete, and
where to look when a citation has to change.
"""
return crud.list_citations(db, _get_or_404(db, reference_id))
@router.patch(
"/{reference_id}", response_model=ReferenceRead, summary="Update a reference"
)
def update_reference(
reference_id: int,
payload: ReferenceUpdate,
db: Session = Depends(get_db),
) -> ReferenceRead:
"""Update any part of a reference.
``authors`` is a full replacement when present — the citation prints them in
the order sent, so there is no partial form of that list.
"""
reference = _get_or_404(db, reference_id)
return crud.read(db, crud.update(db, reference, payload))
@router.delete(
"/{reference_id}",
status_code=status.HTTP_204_NO_CONTENT,
summary="Delete a reference",
)
def delete_reference(reference_id: int, db: Session = Depends(get_db)) -> Response:
"""Delete a reference no sentence cites."""
reference = _get_or_404(db, reference_id)
_assert_not_cited(db, [reference])
crud.delete(db, reference)
return Response(status_code=status.HTTP_204_NO_CONTENT)
+2 -2
View File
@@ -5,6 +5,6 @@ models. They own the commit: a route calls one function and gets back either a
persisted object or ``None``. persisted object or ``None``.
""" """
from app.crud import paper, template, template_title from app.crud import paper, reference, template, template_title
__all__ = ["paper", "template", "template_title"] __all__ = ["paper", "reference", "template", "template_title"]
+26
View File
@@ -31,6 +31,7 @@ from sqlalchemy import func, or_, select
from sqlalchemy import update as sa_update from sqlalchemy import update as sa_update
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
from app.crud import reference as reference_crud
from app.crud.filters import LIKE_ESCAPE, like_pattern from app.crud.filters import LIKE_ESCAPE, like_pattern
from app.models import ( from app.models import (
Paper, Paper,
@@ -528,6 +529,24 @@ def delete_many(db: Session, paper_ids: Sequence[int]) -> int:
# --- paragraphs and sentences ------------------------------------------------ # --- paragraphs and sentences ------------------------------------------------
def _assert_citations_known(
db: Session, sentences: Sequence[object]
) -> None:
"""Check every ``reference_id`` sent with a sentence against the library.
One place for the check, called by each write path, so a new entry point
cannot skip it and let the foreign key be the thing that complains.
"""
wanted = [
citation.reference_id
for sentence in sentences
for citation in getattr(sentence, "citations", []) or []
if citation.reference_id is not None
]
if wanted:
reference_crud.assert_exist(db, wanted)
def replace_paragraph( def replace_paragraph(
db: Session, db: Session,
paper: Paper, paper: Paper,
@@ -546,6 +565,8 @@ def replace_paragraph(
Blank lines are kept if the client sends them: an empty sentence is a Blank lines are kept if the client sends them: an empty sentence is a
legitimate placeholder and renders as a blank line rather than vanishing. legitimate placeholder and renders as a blank line rather than vanishing.
""" """
_assert_citations_known(db, payload.sentences)
target = payload.target_sort if payload.target_sort is not None else field_sort target = payload.target_sort if payload.target_sort is not None else field_sort
# Clear the source paragraph first, so that a move onto itself does not # Clear the source paragraph first, so that a move onto itself does not
@@ -583,6 +604,8 @@ def replace_paragraph(
def append_sentence(db: Session, paper: Paper, data: SentenceCreate) -> PaperSentence: def append_sentence(db: Session, paper: Paper, data: SentenceCreate) -> PaperSentence:
"""Add one sentence to a paragraph, at the end unless told otherwise.""" """Add one sentence to a paragraph, at the end unless told otherwise."""
_assert_citations_known(db, [data])
sort = ( sort = (
data.sort data.sort
if data.sort is not None if data.sort is not None
@@ -614,6 +637,9 @@ def update_sentence(
list is short, and replacing it wholesale is the only way to be sure that a list is short, and replacing it wholesale is the only way to be sure that a
deleted citation really is gone. deleted citation really is gone.
""" """
if data.citations is not None:
_assert_citations_known(db, [data])
if data.sort is not None: if data.sort is not None:
sentence.sort = data.sort sentence.sort = data.sort
if data.paper_template_filed_sort is not None: if data.paper_template_filed_sort is not None:
+375
View File
@@ -0,0 +1,375 @@
"""Data access for the reference library (参考文献).
The list endpoint answers the question a library is opened to answer — "what do
I have, and what is already cited?" — in one query: each row carries its printed
author line and how many sentences cite it, both as correlated subqueries, so a
table row never has to load the rows behind it.
Writes keep the two lists (authors, citations) on their own side: authors belong
to the reference and are replaced wholesale, citations belong to the papers that
made them and are never touched from here.
"""
from collections.abc import Sequence
from sqlalchemy import Select, func, or_, select
from sqlalchemy.orm import Session
from app.crud.filters import LIKE_ESCAPE, like_pattern
from app.models import (
Paper,
PaperSentence,
PaperSentenceReference,
Reference,
ReferenceAuthor,
TemplateTitle,
TemplateTitlePlacement,
)
from app.schemas.reference import (
ReferenceCitationRead,
ReferenceCreate,
ReferenceListItem,
ReferenceRead,
ReferenceUpdate,
)
#: How many authors a table row prints before it cuts the line short.
_LIST_AUTHOR_LIMIT = 3
def _conditions(
keyword: str | None,
type_: str | None,
year: int | None,
cited_only: bool | None,
) -> list:
"""Translate the list filters into SQLAlchemy predicates."""
conditions = []
if keyword:
pattern = like_pattern(keyword)
# The author search is an EXISTS rather than a join: joining would
# duplicate a reference with four authors into four rows.
conditions.append(
or_(
Reference.title.like(pattern, escape=LIKE_ESCAPE),
Reference.subtitle.like(pattern, escape=LIKE_ESCAPE),
Reference.container_title.like(pattern, escape=LIKE_ESCAPE),
Reference.doi.like(pattern, escape=LIKE_ESCAPE),
Reference.keywords.like(pattern, escape=LIKE_ESCAPE),
Reference.note.like(pattern, escape=LIKE_ESCAPE),
select(ReferenceAuthor.id)
.where(
ReferenceAuthor.reference_id == Reference.id,
ReferenceAuthor.name.like(pattern, escape=LIKE_ESCAPE),
)
.exists(),
)
)
if type_:
conditions.append(Reference.type == type_)
if year is not None:
conditions.append(Reference.publication_year == year)
if cited_only is True:
conditions.append(_citation_count_column() > 0)
if cited_only is False:
conditions.append(_citation_count_column() == 0)
return conditions
def _citation_count_column():
"""Correlated count of the sentences that cite this reference."""
return (
select(func.count(PaperSentenceReference.id))
.where(PaperSentenceReference.reference_id == Reference.id)
.correlate(Reference)
.scalar_subquery()
)
def _ordered(stmt: Select) -> Select:
"""Newest publication first, then most recently edited.
A library is browsed by "what did I read about this", so the year leads and
the edit time breaks ties — a reference added today to an older paper still
surfaces if its year is the newest.
"""
return stmt.order_by(
Reference.publication_year.desc(),
Reference.updated_at.desc(),
Reference.id.desc(),
)
def authors_line(authors: Sequence[ReferenceAuthor]) -> str:
"""The printed author line, joined for a table row.
Truncated for display only — the stored list is never cut — because a
fifteen-author row would push every other column off the screen.
"""
names = [author.name for author in authors]
if len(names) > _LIST_AUTHOR_LIMIT:
return ", ".join(names[:_LIST_AUTHOR_LIMIT]) + f" 等 {len(names)} 人"
return ", ".join(names)
def list_references(
db: Session,
*,
keyword: str | None = None,
type_: str | None = None,
year: int | None = None,
cited_only: bool | None = None,
page: int = 1,
page_size: int = 20,
) -> tuple[list[ReferenceListItem], int]:
"""Return one page of the library, plus the unpaged total."""
conditions = _conditions(keyword, type_, year, cited_only)
total = db.scalar(select(func.count(Reference.id)).where(*conditions)) or 0
stmt = _ordered(
select(Reference)
.where(*conditions)
.offset((page - 1) * page_size)
.limit(page_size)
)
rows = list(db.scalars(stmt).all())
# One query for the citation counts of the whole page, rather than one per
# row: the count is a correlated subquery in the list, but here the ids are
# already known and a grouped count is cheaper.
counts = citation_counts(db, [row.id for row in rows])
items = [
ReferenceListItem(
id=row.id,
type=row.type,
title=row.title,
subtitle=row.subtitle,
container_title=row.container_title,
publication_year=row.publication_year,
volume=row.volume,
issue=row.issue,
pages=row.pages,
doi=row.doi,
authors_line=authors_line(row.authors),
author_count=len(row.authors),
citation_count=counts.get(row.id, 0),
created_at=row.created_at,
updated_at=row.updated_at,
)
for row in rows
]
return items, total
def citation_counts(db: Session, reference_ids: Sequence[int]) -> dict[int, int]:
"""How many sentences cite each reference."""
wanted = list({reference_id for reference_id in reference_ids})
if not wanted:
return {}
stmt = (
select(PaperSentenceReference.reference_id, func.count(PaperSentenceReference.id))
.where(PaperSentenceReference.reference_id.in_(wanted))
.group_by(PaperSentenceReference.reference_id)
)
return {reference_id: count for reference_id, count in db.execute(stmt).all()}
def get(db: Session, reference_id: int) -> Reference | None:
"""Return one reference with its authors loaded, or ``None``."""
return db.get(Reference, reference_id)
def get_many(db: Session, reference_ids: Sequence[int]) -> list[Reference]:
"""Return every reference whose id is in ``reference_ids``."""
if not reference_ids:
return []
return list(
db.scalars(select(Reference).where(Reference.id.in_(list(reference_ids)))).all()
)
class UnknownReference(ValueError):
"""A write named literature the library does not have.
A domain error rather than an HTTP one: the same refusal has to reach a
REST client as ``400`` and a model through the MCP tools as a readable tool
error, and the layer that knows what went wrong is this one. Each caller
translates it into its own vocabulary.
"""
def __init__(self, missing: Sequence[int]) -> None:
self.missing = list(missing)
joined = "、".join(str(reference_id) for reference_id in self.missing)
super().__init__(f"参考文献不存在:{joined}")
def assert_exist(db: Session, reference_ids: Sequence[int]) -> None:
"""Refuse a write that cites literature the library does not have.
Called from the paper side before it stores a citation, so the failure is a
message naming the ids rather than a foreign-key violation from the driver.
"""
missing = missing_ids(db, reference_ids)
if missing:
raise UnknownReference(missing)
def missing_ids(db: Session, reference_ids: Sequence[int]) -> list[int]:
"""Which of ``reference_ids`` are not in the library.
Returning the offenders rather than a boolean lets the API name them, which
is the difference between a usable error and "invalid request". A citation
is rejected against this list, so a sentence cannot point at literature that
does not exist.
"""
wanted = set(reference_ids)
if not wanted:
return []
found = set(db.scalars(select(Reference.id).where(Reference.id.in_(wanted))).all())
return sorted(wanted - found)
def read(db: Session, reference: Reference) -> ReferenceRead:
"""Build the detail payload for one reference."""
counts = citation_counts(db, [reference.id])
return ReferenceRead.model_validate(
{
**{
column.name: getattr(reference, column.name)
for column in Reference.__table__.columns
},
"authors": reference.authors,
"citation_count": counts.get(reference.id, 0),
}
)
def _build_authors(
items: Sequence,
) -> list[ReferenceAuthor]:
"""Materialise the client's contributor list, numbering it from 1."""
return [
ReferenceAuthor(
name=item.name,
family_name=item.family_name,
given_name=item.given_name,
role=item.role,
sort=item.sort if item.sort is not None else index,
)
for index, item in enumerate(items, start=1)
]
def create(db: Session, data: ReferenceCreate) -> Reference:
"""Insert a reference together with its contributors."""
values = data.model_dump(exclude={"authors"})
reference = Reference(**values)
reference.authors = _build_authors(data.authors)
db.add(reference)
db.commit()
db.refresh(reference)
return reference
def update(
db: Session,
reference: Reference,
data: ReferenceUpdate,
) -> Reference:
"""Apply a partial update.
``exclude_unset`` is what makes PATCH semantics work: a key the client did
not send leaves the column alone, so clearing a field is an explicit
``null`` and omitting it is not.
"""
values = data.model_dump(exclude_unset=True, exclude={"authors"})
for key, value in values.items():
setattr(reference, key, value)
if data.authors is not None:
# delete-orphan removes the dropped rows on flush; the ORM cascade is
# what cleans them up, whether or not the database would.
reference.authors.clear()
db.flush()
reference.authors.extend(_build_authors(data.authors))
db.commit()
db.refresh(reference)
return reference
def delete(db: Session, reference: Reference) -> None:
"""Delete a reference. Callers must check :func:`citation_counts` first."""
db.delete(reference)
db.commit()
def delete_many(db: Session, reference_ids: Sequence[int]) -> int:
"""Delete several references, returning how many actually existed."""
references = get_many(db, reference_ids)
for reference in references:
db.delete(reference)
db.commit()
return len(references)
def list_citations(
db: Session, reference: Reference
) -> list[ReferenceCitationRead]:
"""Every sentence that cites this reference, with its place in the paper.
The paragraph heading is resolved through the paper's template, so the
answer reads as "in 3.1 Problem Formulation of <paper>" rather than as a
position number only the database understands.
"""
stmt = (
select(PaperSentenceReference, PaperSentence, Paper)
.join(PaperSentence, PaperSentence.id == PaperSentenceReference.sentence_id)
.join(Paper, Paper.id == PaperSentence.paper_id)
.where(PaperSentenceReference.reference_id == reference.id)
.order_by(Paper.id.asc(), PaperSentence.paper_template_filed_sort, PaperSentence.sort)
)
results: list[ReferenceCitationRead] = []
for citation, sentence, paper in db.execute(stmt).all():
results.append(
ReferenceCitationRead(
citation_id=citation.id,
sentence_id=sentence.id,
sentence_content=sentence.content,
quote=citation.quote,
paper_id=paper.id,
paper_title=paper.title,
paper_template_filed_sort=sentence.paper_template_filed_sort,
paragraph_name=_paragraph_name(
db, paper.template_id, sentence.paper_template_filed_sort
),
)
)
return results
def _paragraph_name(db: Session, template_id: int | None, position: int) -> str | None:
"""The heading the template puts at ``position``, or ``None``.
``None`` rather than a placeholder: the document renderer decides how to
print a position the template does not define (it shows 未设定), and this
should not invent a second answer.
"""
if template_id is None:
return None
return db.scalar(
select(TemplateTitle.name)
.join(
TemplateTitlePlacement,
TemplateTitlePlacement.title_id == TemplateTitle.id,
)
.where(
TemplateTitlePlacement.template_id == template_id,
TemplateTitlePlacement.sort == position,
)
.order_by(TemplateTitlePlacement.sort.asc(), TemplateTitlePlacement.id.asc())
.limit(1)
)
+17 -1
View File
@@ -5,11 +5,13 @@ Run locally with::
uvicorn app.main:app --reload --port 8000 uvicorn app.main:app --reload --port 8000
""" """
from fastapi import FastAPI from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from app.api.router import api_router from app.api.router import api_router
from app.core.config import get_settings from app.core.config import get_settings
from app.crud.reference import UnknownReference
settings = get_settings() settings = get_settings()
@@ -27,3 +29,17 @@ app.add_middleware(
) )
app.include_router(api_router, prefix=settings.api_prefix) app.include_router(api_router, prefix=settings.api_prefix)
@app.exception_handler(UnknownReference)
async def unknown_reference_handler(
request: Request, error: UnknownReference
) -> JSONResponse:
"""Answer a citation to literature that is not in the library.
The refusal is raised deep in the data layer, where the ids are known, and
turned into the API's own vocabulary here rather than at every route that
might write a citation. The body matches what the rest of the API returns
for a rejected request, so a client needs no special case.
"""
return JSONResponse(status_code=400, content={"detail": str(error)})
+16 -2
View File
@@ -25,10 +25,14 @@ the module docstring instead, where it costs nothing per request.
from __future__ import annotations from __future__ import annotations
from functools import wraps
from collections.abc import Callable from collections.abc import Callable
from typing import Any, TypeVar from typing import Any, TypeVar
from mcp.server.mcpserver import MCPServer from mcp.server.mcpserver import MCPServer
from mcp.server.mcpserver.exceptions import ToolError
from app.crud.reference import UnknownReference
F = TypeVar("F", bound=Callable[..., Any]) F = TypeVar("F", bound=Callable[..., Any])
@@ -42,13 +46,23 @@ def mcp_tool(
"""Register one tool as unstructured text output.""" """Register one tool as unstructured text output."""
def decorator(fn: F) -> F: def decorator(fn: F) -> F:
@wraps(fn)
def wrapper(*args: Any, **kwargs: Any) -> Any:
# A refusal from the data layer becomes a message the model can read
# and retry against. Left unhandled it would reach the client as
# "Error executing tool ...", which says nothing about what to fix.
try:
return fn(*args, **kwargs)
except UnknownReference as error:
raise ToolError(str(error)) from error
server.add_tool( server.add_tool(
fn, wrapper,
name=name or fn.__name__, name=name or fn.__name__,
title=title, title=title,
structured_output=False, structured_output=False,
) )
return fn return wrapper # type: ignore[return-value]
return decorator return decorator
+12
View File
@@ -35,6 +35,13 @@ from app.models.paper import (
) )
from app.models.paper_sentence import PaperSentence from app.models.paper_sentence import PaperSentence
from app.models.paper_sentence_reference import PaperSentenceReference from app.models.paper_sentence_reference import PaperSentenceReference
from app.models.reference import (
REFERENCE_AUTHOR_ROLES,
REFERENCE_LANGUAGES,
REFERENCE_TYPES,
Reference,
)
from app.models.reference_author import ReferenceAuthor
from app.models.template import Template from app.models.template import Template
from app.models.template_title_placement import TemplateTitlePlacement from app.models.template_title_placement import TemplateTitlePlacement
from app.models.template_title import TemplateTitle from app.models.template_title import TemplateTitle
@@ -47,6 +54,11 @@ __all__ = [
"Paper", "Paper",
"PaperSentence", "PaperSentence",
"PaperSentenceReference", "PaperSentenceReference",
"REFERENCE_AUTHOR_ROLES",
"REFERENCE_LANGUAGES",
"REFERENCE_TYPES",
"Reference",
"ReferenceAuthor",
"Template", "Template",
"TemplateTitlePlacement", "TemplateTitlePlacement",
"TemplateTitle", "TemplateTitle",
+242
View File
@@ -0,0 +1,242 @@
"""The reference library (参考文献库) — the literature a paper cites.
One row is one document the writer has read and may cite: a journal article, a
conference paper, a book, a patent, a standard, a dataset. It is **global**, not
owned by a paper, because the same article is cited from several papers — and
the citation itself (which sentence quotes which passage) belongs to the paper,
in :class:`~app.models.paper_sentence_reference.PaperSentenceReference`.
Why these columns
-----------------
The field set is the intersection of the formats a Chinese journal paper
actually has to survive, so that one row can be rendered as any of them without
a second home for anything:
* **CSL JSON** (``csl-data.json``, 103 fields / 45 types) — what Zotero,
Mendeley and Pandoc exchange. Our names follow it wherever a field is the
same field: ``title``, ``container-title``, ``volume``, ``issue``, ``page``,
``publisher``, ``publisher-place``, ``edition``, ``DOI``, ``ISBN``,
``ISSN``, ``URL``, ``abstract``, ``keyword``, ``language``, ``note``,
``citation-key``, ``event-title``, ``event-place``, ``issued``, ``accessed``.
* **GB/T 7714-2015** — the Chinese national standard a domestic journal will
demand. It is why every entry carries a ``type`` with a marker, and why
patents need ``patent_country`` and ``patent_number``, standards need
``standard_number``, and online resources need ``accessed_date``.
* **BibTeX / RIS** — the export formats a reference manager is judged by.
``citation_key`` is the BibTeX key; the rest map one-to-one with the fields
above.
Contributors are **not** columns: a document has an ordered list of them, each
with a role, which is what :class:`~app.models.reference_author.ReferenceAuthor`
holds. Formats disagree about names anyway — 「张三」, ``Zhang San``,
``Zhang, S.`` are the same person in three styles — so the printed form and the
two halves are stored separately and a style picks what it needs.
What is deliberately missing
----------------------------
The 103 CSL fields include roles this project will never print (narrator,
performer, chair, illustrator, curator…) and archival fields (call-number,
archive-place). Adding a column for each would be a schema nobody reads. They
can arrive when a real citation needs one.
"""
from datetime import date
from typing import TYPE_CHECKING
from sqlalchemy import Date, Index, Integer, SmallInteger, String, Text, text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.db.base import Base
from app.models.mixins import TimestampMixin
if TYPE_CHECKING: # pragma: no cover - typing only
from app.models.reference_author import ReferenceAuthor
#: Document types this library holds, with the marker each format uses for it.
#:
#: ====================== ================ ========= ==========================
#: our value CSL type GB/T [X] example
#: ====================== ================ ========= ==========================
#: journal-article article-journal [J] 期刊论文
#: book book [M] 专著
#: book-chapter chapter [M] 析出文献(书里的一章)
#: conference-paper paper-conference [C] 会议论文
#: thesis thesis [D] 学位论文
#: patent patent [P] 专利
#: standard standard [S] 标准
#: report report [R] 报告
#: newspaper-article article-newspaper[N] 报纸文章
#: web-page webpage [EB/OL] 网页/在线资源
#: dataset dataset [DS] 数据集
#: software software [CP] 计算机程序
#: preprint article [EB/OL] 预印本(arXiv 等)
#: other document [Z] 其他
#: ====================== ================ ========= ==========================
REFERENCE_TYPES: tuple[str, ...] = (
"journal-article",
"book",
"book-chapter",
"conference-paper",
"thesis",
"patent",
"standard",
"report",
"newspaper-article",
"web-page",
"dataset",
"software",
"preprint",
"other",
)
#: The roles a contributor can have. ``author`` is the default and the one that
#: prints in the citation; the others carry the 编 / 译 / 主编 marks that GB/T
#: 7714 puts after a name.
REFERENCE_AUTHOR_ROLES: tuple[str, ...] = (
"author",
"editor",
"translator",
"compiler",
)
#: Languages a row is written in — used to choose which title is primary and
#: which is the 并列题名.
REFERENCE_LANGUAGES: tuple[str, ...] = ("zh", "en", "ja", "other")
class Reference(TimestampMixin, Base):
"""One document in the reference library."""
__tablename__ = "reference"
__table_args__ = (
# The list endpoint filters by type and orders by year, which this
# serves. There is deliberately no index on ``title``: the search box
# matches with a leading wildcard, which no B-tree can use.
Index("ix_reference_type_year", "type", "publication_year"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
#: One of :data:`REFERENCE_TYPES`. Stored as the token, never the Chinese
#: label, so the wording can change without a migration.
type: Mapped[str] = mapped_column(
String(32),
nullable=False,
default="journal-article",
server_default=text("'journal-article'"),
)
# --- the document's name ------------------------------------------------
#: 题名, as printed.
title: Mapped[str] = mapped_column(String(512), nullable=False)
#: 其他题名信息 — the part after a colon (": 基于……的实证研究").
subtitle: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: 并列题名 / 译名 — the title in the other language, which a bilingual
#: manuscript prints alongside the first.
title_translated: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: Short form for in-text citations (CSL ``title-short``).
title_short: Mapped[str | None] = mapped_column(String(255), nullable=True)
# --- where it was published --------------------------------------------
#: 刊名 / 书名 / 论文集名 / 报纸名 — the container the entry sits in.
container_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: 刊名缩写 (CSL ``container-title-short``). Some styles print only this.
container_title_short: Mapped[str | None] = mapped_column(
String(255), nullable=True
)
#: 丛书 / 系列名, and its number.
series_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
series_number: Mapped[str | None] = mapped_column(String(32), nullable=True)
#: 版本项 — "3版", "2nd ed.".
edition: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 卷 and 期. Strings rather than integers: journals print "12(Suppl.)",
#: "S1" and "春季号", and a citation has to reproduce what is printed.
volume: Mapped[str | None] = mapped_column(String(32), nullable=True)
issue: Mapped[str | None] = mapped_column(String(32), nullable=True)
#: 页码, as printed — "1024-1032", "e12345", "12-15, 20".
pages: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 文章编号 / eLocation, for journals that have no page numbers.
article_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- publisher ----------------------------------------------------------
#: 出版者. For a thesis this is the 保存者 (the university), for a report the
#: issuing body, for a standard the publisher — the same role in three
#: formats, which is why there is one column and not three.
publisher: Mapped[str | None] = mapped_column(String(255), nullable=True)
#: 出版地.
publisher_place: Mapped[str | None] = mapped_column(String(255), nullable=True)
# --- events (conferences) ----------------------------------------------
#: 会议名称 / 会议地点 / 会期. GB/T prints the proceedings title, APA and
#: IEEE print the meeting, and a conference paper usually needs both.
event_title: Mapped[str | None] = mapped_column(String(512), nullable=True)
event_place: Mapped[str | None] = mapped_column(String(255), nullable=True)
#: Free text on purpose: conferences run "2024-05-01/03" and styles print
#: the range as written.
event_date: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- dates --------------------------------------------------------------
# Partial dates, exactly as CSL models ``issued``: a book has a year, a
# newspaper article has a day, a patent has its 公告日期.
publication_year: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
publication_month: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
publication_day: Mapped[int | None] = mapped_column(SmallInteger, nullable=True)
#: 引用日期 — required by GB/T 7714 for anything online, because a URL's
#: content can change after it was read.
accessed_date: Mapped[date | None] = mapped_column(Date, nullable=True)
# --- identifiers --------------------------------------------------------
#: Canonical DOI, without the ``https://doi.org/`` prefix.
doi: Mapped[str | None] = mapped_column(String(255), nullable=True)
isbn: Mapped[str | None] = mapped_column(String(32), nullable=True)
issn: Mapped[str | None] = mapped_column(String(32), nullable=True)
url: Mapped[str | None] = mapped_column(String(1024), nullable=True)
#: 专利国别 + 专利号 — GB/T 7714 prints both inside the [P] marker.
patent_country: Mapped[str | None] = mapped_column(String(64), nullable=True)
patent_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: 标准编号 ("GB/T 7714-2015") and 报告编号.
standard_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
report_number: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: Preprint identifier ("arXiv:2405.12345"), which is how an AI paper is
#: cited before it has a DOI.
arxiv_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# --- the writer's own notes --------------------------------------------
abstract: Mapped[str | None] = mapped_column(Text, nullable=True)
#: 文献关键词, comma-separated like the paper's own keywords.
keywords: Mapped[str | None] = mapped_column(String(512), nullable=True)
#: Free text: why it was kept, what it is for, where the PDF is.
note: Mapped[str | None] = mapped_column(Text, nullable=True)
# --- library metadata ---------------------------------------------------
#: The BibTeX key (CSL ``citation-key``), for export.
citation_key: Mapped[str | None] = mapped_column(String(64), nullable=True)
#: One of :data:`REFERENCE_LANGUAGES`; decides which title leads.
language: Mapped[str | None] = mapped_column(String(16), nullable=True)
#: The contributors, in the order the citation prints them.
authors: Mapped[list["ReferenceAuthor"]] = relationship(
back_populates="reference",
cascade="all, delete-orphan",
order_by="ReferenceAuthor.sort, ReferenceAuthor.id",
lazy="selectin",
)
def __repr__(self) -> str: # pragma: no cover - debugging aid
return f"<Reference id={self.id} type={self.type} title={self.title!r}>"
+82
View File
@@ -0,0 +1,82 @@
"""The contributors of one reference (文献责任者).
A citation prints its authors in a fixed order, with a mark for anyone who is
not the author — 「张三, 编」, "Ed. by Smith" — so the list is a child table
rather than a column, and each row knows its position and its role.
Why a name is three fields
--------------------------
Citation styles rewrite names, and they disagree:
=========== ================ ==================
style 「张三」 "John Smith"
=========== ================ ==================
GB/T 7714 张三 Smith J
APA Zhang, S. Smith, J.
IEEE S. Zhang J. Smith
=========== ================ ==================
So the printed form and the two halves are kept apart: ``name`` is what the
writer typed and what prints when a style does not need to rearrange anything,
while ``family_name`` / ``given_name`` are what a style needs to invert or
initialise a name. Both are optional enrichment — a row with only ``name`` is
complete — and that is deliberate rather than duplication: ``name`` is the
authority for display, the halves are the authority for reformatting, and a
style uses whichever it needs. Institutional authors ("世界卫生组织") have no
halves at all, which is why they are not required.
"""
from typing import TYPE_CHECKING
from sqlalchemy import ForeignKey, Index, Integer, String, text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.db.base import Base
if TYPE_CHECKING: # pragma: no cover - typing only
from app.models.reference import Reference
class ReferenceAuthor(Base):
"""One contributor of one reference, at one position, with one role."""
__tablename__ = "reference_author"
__table_args__ = (
# Reads are always "the contributors of this reference, in order".
Index("ix_reference_author_reference_sort", "reference_id", "sort"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
reference_id: Mapped[int] = mapped_column(
ForeignKey("reference.id", ondelete="CASCADE"),
nullable=False,
)
#: Print order, ascending. Ties are broken by ``id``, as everywhere else.
sort: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
#: What prints: 「张三」, "John Smith", "世界卫生组织".
name: Mapped[str] = mapped_column(String(255), nullable=False)
#: Optional halves, for styles that invert or initialise a name.
family_name: Mapped[str | None] = mapped_column(String(128), nullable=True)
given_name: Mapped[str | None] = mapped_column(String(128), nullable=True)
#: One of :data:`app.models.reference.REFERENCE_AUTHOR_ROLES`. The default
#: is the one that needs no mark in the citation.
role: Mapped[str] = mapped_column(
String(16),
nullable=False,
default="author",
server_default=text("'author'"),
)
reference: Mapped["Reference"] = relationship(back_populates="authors")
def __repr__(self) -> str: # pragma: no cover - debugging aid
return (
f"<ReferenceAuthor reference_id={self.reference_id} "
f"sort={self.sort} name={self.name!r} role={self.role}>"
)
+18
View File
@@ -23,6 +23,16 @@ from app.schemas.paper import (
SentenceRead, SentenceRead,
SentenceUpdate, SentenceUpdate,
) )
from app.schemas.reference import (
ReferenceAuthorInput,
ReferenceAuthorRead,
ReferenceCitationRead,
ReferenceCreate,
ReferenceListItem,
ReferenceRead,
ReferenceUpdate,
normalize_doi,
)
from app.schemas.template import ( from app.schemas.template import (
TemplateCreate, TemplateCreate,
TemplateListItem, TemplateListItem,
@@ -40,6 +50,14 @@ from app.schemas.template_title import (
__all__ = [ __all__ = [
"BatchDeleteRequest", "BatchDeleteRequest",
"ReferenceAuthorInput",
"ReferenceAuthorRead",
"ReferenceCitationRead",
"ReferenceCreate",
"ReferenceListItem",
"ReferenceRead",
"ReferenceUpdate",
"normalize_doi",
"CitationInput", "CitationInput",
"CitationRead", "CitationRead",
"PaperCitationRead", "PaperCitationRead",
+406
View File
@@ -0,0 +1,406 @@
"""Request/response schemas for the reference library (参考文献).
The field set and where it comes from is documented on
:mod:`app.models.reference`; this module is about the wire format:
* **Nothing is stored that a style can derive.** ``authors`` arrives as an
ordered list and replaces the stored list wholesale, like a template's
placements — the citation prints them in exactly the order sent.
* **A DOI is stored canonically.** ``https://doi.org/10.1000/xyz``,
``doi:10.1000/xyz`` and ``10.1000/xyz`` are the same DOI, so the prefix is
stripped on write and the column holds one spelling.
* **A date is as precise as it is known.** Year, month and day are separate and
independently optional, which is what CSL calls a partial date: a book has a
year, a newspaper article a day, a patent its 公告日期.
"""
from datetime import date, datetime
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
from app.models.reference import (
REFERENCE_AUTHOR_ROLES,
REFERENCE_LANGUAGES,
REFERENCE_TYPES,
)
#: Prefixes people paste along with a DOI. Stripped on write so the column has
#: one spelling, and so a citation builder never has to test for them.
_DOI_PREFIXES = (
"https://doi.org/",
"http://doi.org/",
"https://dx.doi.org/",
"http://dx.doi.org/",
"doi:",
"DOI:",
)
def normalize_doi(value: str | None) -> str | None:
"""Return the bare DOI, or ``None`` when nothing was given."""
if value is None:
return None
text = value.strip()
for prefix in _DOI_PREFIXES:
if text.lower().startswith(prefix.lower()):
text = text[len(prefix) :]
break
return text.strip() or None
class ReferenceAuthorInput(BaseModel):
"""One contributor as the client sends it.
``name`` is required — it is what prints. ``family_name`` / ``given_name``
are optional enrichment for styles that invert or initialise a name, and a
``sort`` that is left out is assigned from the position in the list.
"""
name: str = Field(min_length=1, max_length=255)
family_name: str | None = Field(default=None, max_length=128)
given_name: str | None = Field(default=None, max_length=128)
role: str = Field(default="author", max_length=16)
sort: int | None = Field(default=None, ge=-1_000_000, le=1_000_000)
@field_validator("name")
@classmethod
def _require_name(cls, value: str) -> str:
text = value.strip()
if not text:
raise ValueError("责任者姓名不能为空")
return text
@field_validator("family_name", "given_name")
@classmethod
def _strip_optional(cls, value: str | None) -> str | None:
if value is None:
return None
return value.strip() or None
@field_validator("role")
@classmethod
def _check_role(cls, value: str) -> str:
if value not in REFERENCE_AUTHOR_ROLES:
raise ValueError(
"责任者角色只能是:" + "、".join(REFERENCE_AUTHOR_ROLES)
)
return value
class ReferenceAuthorRead(BaseModel):
"""A stored contributor."""
model_config = ConfigDict(from_attributes=True)
id: int
sort: int
name: str
family_name: str | None
given_name: str | None
role: str
class ReferenceBase(BaseModel):
"""Every field a reference may carry, all optional but the title."""
type: str = Field(default="journal-article", max_length=32)
title: str = Field(min_length=1, max_length=512)
subtitle: str | None = Field(default=None, max_length=512)
title_translated: str | None = Field(default=None, max_length=512)
title_short: str | None = Field(default=None, max_length=255)
container_title: str | None = Field(default=None, max_length=512)
container_title_short: str | None = Field(default=None, max_length=255)
series_title: str | None = Field(default=None, max_length=512)
series_number: str | None = Field(default=None, max_length=32)
edition: str | None = Field(default=None, max_length=64)
volume: str | None = Field(default=None, max_length=32)
issue: str | None = Field(default=None, max_length=32)
pages: str | None = Field(default=None, max_length=64)
article_number: str | None = Field(default=None, max_length=64)
publisher: str | None = Field(default=None, max_length=255)
publisher_place: str | None = Field(default=None, max_length=255)
event_title: str | None = Field(default=None, max_length=512)
event_place: str | None = Field(default=None, max_length=255)
event_date: str | None = Field(default=None, max_length=64)
publication_year: int | None = Field(default=None, ge=1, le=2999)
publication_month: int | None = Field(default=None, ge=1, le=12)
publication_day: int | None = Field(default=None, ge=1, le=31)
accessed_date: date | None = None
doi: str | None = Field(default=None, max_length=255)
isbn: str | None = Field(default=None, max_length=32)
issn: str | None = Field(default=None, max_length=32)
url: str | None = Field(default=None, max_length=1024)
patent_country: str | None = Field(default=None, max_length=64)
patent_number: str | None = Field(default=None, max_length=64)
standard_number: str | None = Field(default=None, max_length=64)
report_number: str | None = Field(default=None, max_length=64)
arxiv_id: str | None = Field(default=None, max_length=64)
abstract: str | None = None
keywords: str | None = Field(default=None, max_length=512)
note: str | None = None
citation_key: str | None = Field(default=None, max_length=64)
language: str | None = Field(default=None, max_length=16)
authors: list[ReferenceAuthorInput] = Field(default_factory=list)
@field_validator("title")
@classmethod
def _require_title(cls, value: str) -> str:
text = value.strip()
if not text:
raise ValueError("文献题名不能为空")
return text
@field_validator("type")
@classmethod
def _check_type(cls, value: str) -> str:
if value not in REFERENCE_TYPES:
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
return value
@field_validator("language")
@classmethod
def _check_language(cls, value: str | None) -> str | None:
if value is None or value == "":
return None
if value not in REFERENCE_LANGUAGES:
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
return value
@field_validator("doi")
@classmethod
def _normalise_doi(cls, value: str | None) -> str | None:
return normalize_doi(value)
@field_validator(
"subtitle",
"title_translated",
"title_short",
"container_title",
"container_title_short",
"series_title",
"series_number",
"edition",
"volume",
"issue",
"pages",
"article_number",
"publisher",
"publisher_place",
"event_title",
"event_place",
"event_date",
"isbn",
"issn",
"url",
"patent_country",
"patent_number",
"standard_number",
"report_number",
"arxiv_id",
"keywords",
"citation_key",
)
@classmethod
def _blank_to_none(cls, value: str | None) -> str | None:
"""Treat a whitespace-only string as "not set" rather than as a value."""
if value is None:
return None
return value.strip() or None
@model_validator(mode="after")
def _check_online_resource(self) -> "ReferenceBase":
"""An online resource must say where it is.
GB/T 7714 prints a URL for ``[EB/OL]`` and nothing else identifies the
document; a row typed as a web page or a dataset with no URL is a row
that cannot be cited.
"""
if self.type in ("web-page", "dataset") and not (self.url or self.doi):
raise ValueError("网页/数据集类型的文献必须至少给出 URL 或 DOI")
return self
class ReferenceCreate(ReferenceBase):
"""Payload for ``POST /references``."""
class ReferenceUpdate(BaseModel):
"""Payload for ``PATCH /references/{id}`` — every part optional.
``authors`` is a full replacement when present, like a template's
placements: the citation prints the list in the order it arrives.
"""
type: str | None = Field(default=None, max_length=32)
title: str | None = Field(default=None, min_length=1, max_length=512)
subtitle: str | None = Field(default=None, max_length=512)
title_translated: str | None = Field(default=None, max_length=512)
title_short: str | None = Field(default=None, max_length=255)
container_title: str | None = Field(default=None, max_length=512)
container_title_short: str | None = Field(default=None, max_length=255)
series_title: str | None = Field(default=None, max_length=512)
series_number: str | None = Field(default=None, max_length=32)
edition: str | None = Field(default=None, max_length=64)
volume: str | None = Field(default=None, max_length=32)
issue: str | None = Field(default=None, max_length=32)
pages: str | None = Field(default=None, max_length=64)
article_number: str | None = Field(default=None, max_length=64)
publisher: str | None = Field(default=None, max_length=255)
publisher_place: str | None = Field(default=None, max_length=255)
event_title: str | None = Field(default=None, max_length=512)
event_place: str | None = Field(default=None, max_length=255)
event_date: str | None = Field(default=None, max_length=64)
publication_year: int | None = Field(default=None, ge=1, le=2999)
publication_month: int | None = Field(default=None, ge=1, le=12)
publication_day: int | None = Field(default=None, ge=1, le=31)
accessed_date: date | None = None
doi: str | None = Field(default=None, max_length=255)
isbn: str | None = Field(default=None, max_length=32)
issn: str | None = Field(default=None, max_length=32)
url: str | None = Field(default=None, max_length=1024)
patent_country: str | None = Field(default=None, max_length=64)
patent_number: str | None = Field(default=None, max_length=64)
standard_number: str | None = Field(default=None, max_length=64)
report_number: str | None = Field(default=None, max_length=64)
arxiv_id: str | None = Field(default=None, max_length=64)
abstract: str | None = None
keywords: str | None = Field(default=None, max_length=512)
note: str | None = None
citation_key: str | None = Field(default=None, max_length=64)
language: str | None = Field(default=None, max_length=16)
authors: list[ReferenceAuthorInput] | None = None
@field_validator("title")
@classmethod
def _require_title(cls, value: str | None) -> str | None:
if value is None:
return None
text = value.strip()
if not text:
raise ValueError("文献题名不能为空")
return text
@field_validator("type")
@classmethod
def _check_type(cls, value: str | None) -> str | None:
if value is None:
return None
if value not in REFERENCE_TYPES:
raise ValueError("文献类型只能是:" + "、".join(REFERENCE_TYPES))
return value
@field_validator("language")
@classmethod
def _check_language(cls, value: str | None) -> str | None:
if value is None or value == "":
return None
if value not in REFERENCE_LANGUAGES:
raise ValueError("语种只能是:" + "、".join(REFERENCE_LANGUAGES))
return value
@field_validator("doi")
@classmethod
def _normalise_doi(cls, value: str | None) -> str | None:
return normalize_doi(value)
class ReferenceListItem(BaseModel):
"""A reference as it appears in the library table."""
id: int
type: str
title: str
subtitle: str | None
container_title: str | None
publication_year: int | None
volume: str | None
issue: str | None
pages: str | None
doi: str | None
#: The printed author list, joined — a table row needs the line, not the
#: rows behind it.
authors_line: str
author_count: int
#: How many sentences cite it. Drives the refusal message on delete, and
#: tells the writer at a glance what is already used.
citation_count: int
created_at: datetime
updated_at: datetime
class ReferenceRead(BaseModel):
"""A reference with everything the table does not need."""
id: int
type: str
title: str
subtitle: str | None
title_translated: str | None
title_short: str | None
container_title: str | None
container_title_short: str | None
series_title: str | None
series_number: str | None
edition: str | None
volume: str | None
issue: str | None
pages: str | None
article_number: str | None
publisher: str | None
publisher_place: str | None
event_title: str | None
event_place: str | None
event_date: str | None
publication_year: int | None
publication_month: int | None
publication_day: int | None
accessed_date: date | None
doi: str | None
isbn: str | None
issn: str | None
url: str | None
patent_country: str | None
patent_number: str | None
standard_number: str | None
report_number: str | None
arxiv_id: str | None
abstract: str | None
keywords: str | None
note: str | None
citation_key: str | None
language: str | None
authors: list[ReferenceAuthorRead]
citation_count: int
created_at: datetime
updated_at: datetime
class ReferenceCitationRead(BaseModel):
"""One sentence that cites a reference — the reverse lookup.
"Which of my sentences use this article?" is the question a reference
library exists to answer, and it needs the sentence, the paragraph it sits
in, and the paper it belongs to.
"""
citation_id: int
sentence_id: int
sentence_content: str
quote: str
paper_id: int
paper_title: str
paper_template_filed_sort: int
paragraph_name: str | None = None
__all__ = [
"ReferenceAuthorInput",
"ReferenceAuthorRead",
"ReferenceBase",
"ReferenceCitationRead",
"ReferenceCreate",
"ReferenceListItem",
"ReferenceRead",
"ReferenceUpdate",
"normalize_doi",
]
+50 -4
View File
@@ -26,8 +26,8 @@ import argparse
import json import json
import os import os
import sys import sys
from collections.abc import AsyncIterator from collections.abc import AsyncIterator, Iterator
from contextlib import asynccontextmanager from contextlib import asynccontextmanager, contextmanager
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -43,7 +43,7 @@ from mcp.client.stdio import StdioServerParameters, stdio_client # noqa: E402
from sqlalchemy import or_, select # noqa: E402 from sqlalchemy import or_, select # noqa: E402
from app.db.session import SessionLocal # noqa: E402 from app.db.session import SessionLocal # noqa: E402
from app.models import Paper, Template # noqa: E402 from app.models import Paper, Reference, Template # noqa: E402
SERVER_SCRIPT = BACKEND_DIR / "scripts" / "mcp_server.py" SERVER_SCRIPT = BACKEND_DIR / "scripts" / "mcp_server.py"
@@ -97,6 +97,15 @@ async def expect_error(session: ClientSession, tool: str, **arguments: Any) -> s
) )
class _Box:
"""A one-slot holder, so the citation helper can read what the context set."""
value: int | None = None
_CITATION_REFERENCE_ID = _Box()
def pick_template() -> tuple[int, list[dict[str, Any]]]: def pick_template() -> tuple[int, list[dict[str, Any]]]:
"""The template with the most titles, and its outline as the tools see it.""" """The template with the most titles, and its outline as the tools see it."""
with SessionLocal() as db: with SessionLocal() as db:
@@ -145,6 +154,11 @@ async def connect(url: str | None, token: str | None) -> AsyncIterator[ClientSes
async def run(url: str | None = None, token: str | None = None) -> int: async def run(url: str | None = None, token: str | None = None) -> int:
with reference_for_citation():
return await _run(url, token)
async def _run(url: str | None = None, token: str | None = None) -> int:
template_id, outline = pick_template() template_id, outline = pick_template()
check("a template with fields exists", len(outline) >= 3, f"got {len(outline)}") check("a template with fields exists", len(outline) >= 3, f"got {len(outline)}")
first = outline[0]["name"] first = outline[0]["name"]
@@ -257,7 +271,7 @@ async def run(url: str | None = None, token: str | None = None) -> int:
paper_id=paper["id"], paper_id=paper["id"],
heading=first, heading=first,
content="A cited sentence.", content="A cited sentence.",
citations=[{"quote": "引用的原话", "reference_id": 7}], citations=[{"quote": "引用的原话", "reference_id": _reference_id()}],
) )
check("appended", added["sentence"]["content"] == "A cited sentence.") check("appended", added["sentence"]["content"] == "A cited sentence.")
check("the citation is stored", added["sentence"]["citations"][0]["quote"] == "引用的原话") check("the citation is stored", added["sentence"]["citations"][0]["quote"] == "引用的原话")
@@ -436,6 +450,38 @@ async def run(url: str | None = None, token: str | None = None) -> int:
return 0 return 0
@contextmanager
def reference_for_citation() -> Iterator[int]:
"""A library entry the citation checks can point at.
A citation names real literature now, so the test has to have some: the MCP
surface has no reference tools yet, and this creates the row directly.
"""
with SessionLocal() as db:
reference = Reference(type="journal-article", title="SMOKE MCP 文献")
db.add(reference)
db.commit()
db.refresh(reference)
reference_id = reference.id
_CITATION_REFERENCE_ID.value = reference_id
try:
yield reference_id
finally:
with SessionLocal() as db:
stored = db.get(Reference, reference_id)
if stored is not None:
db.delete(stored)
db.commit()
_CITATION_REFERENCE_ID.value = None
def _reference_id() -> int:
"""The id the current citation check should use."""
if _CITATION_REFERENCE_ID.value is None:
raise AssertionError("no reference in scope for this citation check")
return _CITATION_REFERENCE_ID.value
def _field_id(name: str) -> int: def _field_id(name: str) -> int:
"""The library field behind a heading name, for the template-create step.""" """The library field behind a heading name, for the template-create step."""
from app.models import TemplateTitle from app.models import TemplateTitle
+38 -3
View File
@@ -63,6 +63,7 @@ def check(label: str, condition: bool, detail: str = "") -> None:
def main() -> int: def main() -> int:
created_papers: list[int] = [] created_papers: list[int] = []
created_references: list[int] = []
try: try:
# The database may already hold real papers, so "nothing of ours is # The database may already hold real papers, so "nothing of ours is
@@ -117,6 +118,21 @@ def main() -> int:
first_sort = document["paragraphs"][0]["paper_template_filed_sort"] first_sort = document["paragraphs"][0]["paper_template_filed_sort"]
print("4. write a paragraph: two sentences, citations on both") print("4. write a paragraph: two sentences, citations on both")
# A citation names real literature now, so the library needs an entry
# first — and a citation with no id at all stays legal.
reference = call(
"POST",
"/references",
{
"type": "journal-article",
"title": "用于验证引用链路的文献",
"container_title": "测试期刊",
"publication_year": 2024,
"authors": [{"name": "测试作者"}],
},
expect=201,
)
created_references.append(reference["id"])
document = call( document = call(
"PUT", "PUT",
f"/papers/{paper['id']}/paragraphs/{first_sort}", f"/papers/{paper['id']}/paragraphs/{first_sort}",
@@ -124,12 +140,14 @@ def main() -> int:
"sentences": [ "sentences": [
{ {
"content": " 第一句话,用于验证空白折叠。 ", "content": " 第一句话,用于验证空白折叠。 ",
"citations": [{"reference_id": 7, "quote": "被引用的第一段内容"}], "citations": [
{"reference_id": reference["id"], "quote": "被引用的第一段内容"}
],
}, },
{ {
"content": "第二句话。", "content": "第二句话。",
"citations": [ "citations": [
{"reference_id": 9, "quote": "引文甲"}, {"reference_id": reference["id"], "quote": "引文甲"},
{"quote": "只有引用内容、暂时没有引用 id"}, {"quote": "只有引用内容、暂时没有引用 id"},
], ],
}, },
@@ -230,7 +248,16 @@ def main() -> int:
bad = call( bad = call(
"PUT", "PUT",
f"/papers/{paper['id']}/paragraphs/{first_sort}", f"/papers/{paper['id']}/paragraphs/{first_sort}",
{"sentences": [{"content": "x", "citations": [{"reference_id": 3, "quote": " "}]}]}, {
"sentences": [
{
"content": "x",
"citations": [
{"reference_id": reference["id"], "quote": " "}
],
}
]
},
expect=422, expect=422,
) )
check("citation without content refused", "detail" in bad) check("citation without content refused", "detail" in bad)
@@ -361,6 +388,9 @@ def main() -> int:
print("15. delete the paper") print("15. delete the paper")
call("DELETE", f"/papers/{paper['id']}", expect=204) call("DELETE", f"/papers/{paper['id']}", expect=204)
created_papers.clear() created_papers.clear()
for reference_id in created_references:
call("DELETE", f"/references/{reference_id}", expect=204)
created_references.clear()
check( check(
"gone from the list", "gone from the list",
call("GET", "/papers")["total"] == baseline, call("GET", "/papers")["total"] == baseline,
@@ -371,6 +401,11 @@ def main() -> int:
return 0 return 0
finally: finally:
# Never leave test rows behind, even on a failed expectation. # Never leave test rows behind, even on a failed expectation.
for reference_id in created_references:
try:
call("DELETE", f"/references/{reference_id}", expect=204)
except Exception as error: # noqa: BLE001 - cleanup must not mask the failure
print(f"cleanup of reference {reference_id} failed: {error}", file=sys.stderr)
for paper_id in created_papers: for paper_id in created_papers:
try: try:
call("DELETE", f"/papers/{paper_id}", expect=204) call("DELETE", f"/papers/{paper_id}", expect=204)
+260
View File
@@ -0,0 +1,260 @@
"""End-to-end check of the reference library, against a running API.
Run it while the backend is up::
.venv/bin/python scripts/smoke_references.py
It walks the whole loop the library exists for — add literature, find it again
by author and by type, link a sentence to it, look the citation up from the
reference side, be refused when deleting something in use, then clean up — and
exits non-zero on the first failed expectation. Everything it creates is
deleted, including the throwaway paper it cites from.
"""
from __future__ import annotations
import json
import sys
import urllib.error
import urllib.parse
import urllib.request
from typing import Any
BASE = "http://127.0.0.1:8000/api"
_checks = 0
def call(method: str, path: str, body: Any = None, expect: int = 200) -> Any:
"""Perform one API call and assert its status code."""
data = json.dumps(body).encode() if body is not None else None
url = BASE + urllib.parse.quote(path, safe="/?&=%")
request = urllib.request.Request(
url,
data=data,
method=method,
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(request, timeout=20) as response:
status = response.status
payload = response.read().decode()
except urllib.error.HTTPError as error:
status = error.code
payload = error.read().decode()
if status != expect:
raise AssertionError(f"{method} {path} -> {status}, expected {expect}: {payload}")
return json.loads(payload) if payload else None
def check(label: str, condition: bool, detail: str = "") -> None:
"""Assert one expectation and report it."""
global _checks
_checks += 1
if not condition:
raise AssertionError(f"{label}: {detail}")
print(f" ok {label}")
def main() -> int:
created_references: list[int] = []
created_papers: list[int] = []
try:
print("1. add an article with four authors")
article = call(
"POST",
"/references",
{
"type": "journal-article",
"title": "技术惯性与冲量的耦合机制",
"subtitle": "基于科学与技术融合网络的实证分析",
"title_translated": "Coupling inertia and impulse in technology evolution",
"container_title": "Technological Forecasting and Social Change",
"container_title_short": "Technol. Forecast. Soc. Change",
"volume": "198",
"issue": "3",
"pages": "122-141",
"publication_year": 2024,
"publication_month": 5,
"doi": "https://doi.org/10.1016/j.techfore.2024.123456",
"language": "zh",
"keywords": "技术演化, 惯性, 冲量",
"authors": [
{"name": "张三", "family_name": "张", "given_name": "三"},
{"name": "李四"},
{"name": "Wang Wu", "family_name": "Wang", "given_name": "Wu"},
{"name": "世界卫生组织", "role": "compiler"},
],
},
expect=201,
)
created_references.append(article["id"])
check("the DOI lost its URL prefix", article["doi"] == "10.1016/j.techfore.2024.123456", article["doi"])
check("four contributors stored", len(article["authors"]) == 4)
check(
"they keep the order they were sent in",
[a["name"] for a in article["authors"]] == ["张三", "李四", "Wang Wu", "世界卫生组织"],
str([a["name"] for a in article["authors"]]),
)
check(
"sort is numbered from 1",
[a["sort"] for a in article["authors"]] == [1, 2, 3, 4],
str([a["sort"] for a in article["authors"]]),
)
check("roles survive", article["authors"][3]["role"] == "compiler")
check("partial date kept as sent", article["publication_year"] == 2024 and article["publication_month"] == 5)
check("no day invented", article["publication_day"] is None)
check("not cited yet", article["citation_count"] == 0)
print("2. add a patent, which needs a country and a number")
patent = call(
"POST",
"/references",
{
"type": "patent",
"title": "一种基于多层聚类的人工智能技术识别方法",
"patent_country": "中国",
"patent_number": "CN114000000A",
"publication_year": 2023,
"authors": [{"name": "张三"}],
},
expect=201,
)
created_references.append(patent["id"])
check("patent stored", patent["type"] == "patent" and patent["patent_number"] == "CN114000000A")
print("3. a webpage without a URL is refused")
call(
"POST",
"/references",
{"type": "web-page", "title": "某网页"},
expect=422,
)
check("web page without URL refused", True)
print("4. a bad type, a phantom citation target and a missing title are refused")
call("POST", "/references", {"type": "blog-post", "title": "x"}, expect=422)
call("POST", "/references", {"title": " "}, expect=422)
call(
"POST",
"/references",
{"title": "引用了不存在的文献", "authors": [{"name": "甲"}], "type": "book"},
expect=201,
)
probe = call("GET", "/references?keyword=引用了不存在")
created_references.append(probe["items"][0]["id"])
check("a book needs nothing else", probe["total"] == 1)
print("5. find it again: by author, by type, by year")
by_author = call("GET", "/references?keyword=Wang")
check("search matches an author name", by_author["total"] == 1, str(by_author["total"]))
check(
"the row prints an author line",
by_author["items"][0]["authors_line"].startswith("张三, 李四, Wang Wu"),
by_author["items"][0]["authors_line"],
)
by_type = call("GET", "/references?type=patent")
check("filter by type", by_type["total"] == 1, str(by_type["total"]))
by_year = call("GET", "/references?year=2024")
check("filter by year", by_year["total"] == 1, str(by_year["total"]))
uncited = call("GET", "/references?cited=false")
check("filter by not-yet-cited", uncited["total"] == len(created_references))
print("6. edit it: replace the contributor list, change a field")
updated = call(
"PATCH",
f"/references/{article['id']}",
{
"pages": "122-145",
"authors": [
{"name": "张三", "family_name": "张", "given_name": "三"},
{"name": "李四"},
],
},
)
check("pages updated", updated["pages"] == "122-145")
check("author list replaced wholesale", len(updated["authors"]) == 2)
check("untouched fields kept", updated["container_title"] == "Technological Forecasting and Social Change")
print("7. cite it from a sentence")
templates = call("GET", "/templates?page_size=1")["items"]
paper = call(
"POST",
"/papers",
{"title": "参考文献冒烟测试论文", "template_id": templates[0]["id"]},
expect=201,
)
created_papers.append(paper["id"])
outline = call("GET", f"/templates/{templates[0]['id']}")
position = outline["placements"][0]["sort"]
call(
"PUT",
f"/papers/{paper['id']}/paragraphs/{position}",
{
"sentences": [
{
"content": "技术惯性约束了同时具有高冲量的可能。",
"citations": [
{"reference_id": article["id"], "quote": "高惯性技术几乎不可能同时具有高冲量。"}
],
}
]
},
)
check("a citation to a real reference is accepted", True)
missing = call(
"PUT",
f"/papers/{paper['id']}/paragraphs/{position}",
{"sentences": [{"content": "x", "citations": [{"reference_id": 999999, "quote": "y"}]}]},
expect=400,
)
check("a citation to nothing is refused by name", "999999" in missing["detail"], missing["detail"])
print("8. the reverse lookup: which sentences use this article?")
citations = call("GET", f"/references/{article['id']}/citations")
check("one citation found", len(citations) == 1, str(len(citations)))
check("it names the sentence", "技术惯性约束" in citations[0]["sentence_content"])
check("and the paper", citations[0]["paper_id"] == paper["id"])
check("and the paragraph heading", citations[0]["paragraph_name"] is not None, str(citations[0]))
cited = call("GET", "/references?cited=true")
check("the cited filter now finds it", cited["total"] == 1, str(cited["total"]))
print("9. a cited reference cannot be deleted; an uncited one can")
conflict = call("DELETE", f"/references/{article['id']}", expect=409)
check("refusal explains itself", "正被论文引用" in conflict["detail"], conflict["detail"])
call("DELETE", f"/references/{patent['id']}", expect=204)
created_references.remove(patent["id"])
check("an uncited reference deletes cleanly", True)
print("10. clean up")
call("DELETE", f"/papers/{paper['id']}", expect=204)
created_papers.clear()
for reference_id in created_references:
call("DELETE", f"/references/{reference_id}", expect=204)
created_references.clear()
remaining = call("GET", "/references")
check("nothing of ours is left", remaining["total"] == 0, str(remaining["total"]))
print(f"\nall {_checks} checks passed")
return 0
finally:
for paper_id in created_papers:
try:
call("DELETE", f"/papers/{paper_id}", expect=204)
except Exception as error: # noqa: BLE001 - cleanup must not mask the failure
print(f"cleanup of paper {paper_id} failed: {error}", file=sys.stderr)
for reference_id in created_references:
try:
call("DELETE", f"/references/{reference_id}", expect=204)
print(f"cleaned up reference {reference_id}")
except Exception as error: # noqa: BLE001
print(f"cleanup of reference {reference_id} failed: {error}", file=sys.stderr)
if __name__ == "__main__":
sys.exit(main())
+79 -3
View File
@@ -58,8 +58,8 @@ Browser ──HTTP/JSON──▶ FastAPI ──SQLAlchemy──▶ TiDB (k3s)
## Domain Model ## Domain Model
Six tables. Every one is prefixed with the module it belongs to, and a model Nine tables. Every one is prefixed with the module it belongs to, and a model
class is named after its table, so the schema can be read as three groups: class is named after its table, so the schema can be read as four groups:
``` ```
template_title one reusable heading (字段库) template_title one reusable heading (字段库)
@@ -79,7 +79,14 @@ paper_sentence one sentence, at one position, in one paper
id, paper_id → paper, template_id, paper_template_filed_sort, sort, content id, paper_id → paper, template_id, paper_template_filed_sort, sort, content
paper_sentence_reference the citations of one sentence paper_sentence_reference the citations of one sentence
id, sentence_id → paper_sentence, reference_id, quote, sort id, sentence_id → paper_sentence, reference_id → reference, quote, sort
reference the 参考文献 module: the literature itself
id, type, title, subtitle, container_title, volume, issue, pages,
publisher, publisher_place, publication_year, doi, url, …
reference_author its contributors, in print order
id, reference_id → reference, sort, name, family_name, given_name, role
``` ```
### How the tables are named ### How the tables are named
@@ -194,6 +201,49 @@ written against. It is kept in step with the paper's current template and is
provenance rather than a lookup key: rendering never filters on it, which is provenance rather than a lookup key: rendering never filters on it, which is
precisely why content survives a switch. precisely why content survives a switch.
### The reference library holds literature, not citations
``reference`` is the *literature*: one row per document the writer has read, and
it is **global** — the same article is cited from several papers, so the entry
cannot belong to one of them. The *citation* is a different fact — "this
sentence quotes this passage" — and it lives on the paper side, in
``paper_sentence_reference``, which carries the quote and the citation's order
inside the sentence.
So the relation is many-to-many in both directions, through that table:
```
paper_sentence ──< paper_sentence_reference >── reference
(quote, sort) (title, year, …)
```
Two questions follow from it, and each is answered from its own side:
"what does this sentence quote?" is a lookup by ``sentence_id``, and "where have
I used this article?" is a lookup by ``reference_id``
(``GET /references/{id}/citations``). One column on ``paper_sentence`` could not
express the first of those — it would cap a sentence at one citation — which is
why the relation is a table rather than a foreign key.
#### Why these columns and no others
The field set is the intersection of the formats a paper has to survive:
| Format | What it decided |
|---|---|
| CSL JSON (``csl-data.json``: 103 fields, 45 types) | the field names, and partial dates as year / month / day rather than one ``DATE`` |
| GB/T 7714-2015 | every entry carries a ``type`` with its marker; patents need ``patent_country`` + ``patent_number``, standards ``standard_number``, online resources ``accessed_date`` |
| BibTeX / RIS | ``citation_key``, and the one-field-per-fact layout that makes an export a mapping rather than a parse |
Contributors are a child table, not a column, and each row stores the printed
name *and* its two halves. That is not duplication: a citation prints 「张三」,
``Zhang, S.`` or ``S. Zhang`` depending on the style, so the printed form is the
authority for display and the halves are the authority for rewriting a name —
and an institutional author (「世界卫生组织」) has no halves at all.
The 103 CSL fields include roles this project will never print (narrator,
performer, curator) and archival fields (call-number, archive-place). They are
left out on purpose; see :mod:`app.models.reference`.
### A sentence is a line, not parsed text ### A sentence is a line, not parsed text
Nothing in the writing path splits text. There is no sentence detector, no Nothing in the writing path splits text. There is no sentence detector, no
@@ -299,6 +349,11 @@ All routes are mounted under `/api`. Interactive docs at `/docs`.
| `POST` | `/papers` | title + template + author/status/keywords/journal; no sentence rows are created | | `POST` | `/papers` | title + template + author/status/keywords/journal; no sentence rows are created |
| `GET` `PATCH` `DELETE` | `/papers/{id}` | `PATCH` with `template_id` **is** the template switch | | `GET` `PATCH` `DELETE` | `/papers/{id}` | `PATCH` with `template_id` **is** the template switch |
| `POST` | `/papers/batch-delete` | body `{ "ids": [...] }` | | `POST` | `/papers/batch-delete` | body `{ "ids": [...] }` |
| `GET` | `/references` | `keyword` (题名/刊名/作者/DOI/关键词/备注), `type`, `year`, `cited` |
| `POST` | `/references` | the whole entry, `authors` in print order |
| `GET` `PATCH` `DELETE` | `/references/{id}` | `PATCH` with `authors` replaces the list |
| `POST` | `/references/batch-delete` | body `{ "ids": [...] }` |
| `GET` | `/references/{id}/citations` | which sentences cite it, and in which paragraph |
| `GET` | `/papers/{id}/document` | the whole paper: paragraphs in order, citations, warnings | | `GET` | `/papers/{id}/document` | the whole paper: paragraphs in order, citations, warnings |
| `GET` | `/papers/{id}/paragraphs/{sort}` | one paragraph, assembled as the document renders it | | `GET` | `/papers/{id}/paragraphs/{sort}` | one paragraph, assembled as the document renders it |
| `PUT` | `/papers/{id}/paragraphs/{sort}` | full replacement; optional `target_sort` moves it | | `PUT` | `/papers/{id}/paragraphs/{sort}` | full replacement; optional `target_sort` moves it |
@@ -327,6 +382,14 @@ Conventions worth knowing:
- `keywords` is stored as a canonical ``, ``-joined string; `,`、`;` and `;` - `keywords` is stored as a canonical ``, ``-joined string; `,`、`;` and `;`
are accepted on write and de-duplicated. are accepted on write and de-duplicated.
- Paper `status` is one of `draft`, `writing`, `done`. - Paper `status` is one of `draft`, `writing`, `done`.
- A citation's ``reference_id`` is checked against the library: citing
literature that does not exist is a `400` naming the ids, not a foreign-key
error from the driver. A citation with no ``reference_id`` stays legal — a
quote may be written before it is linked.
- Deleting a reference a sentence cites is refused with `409`, naming the entry
and how many citations use it.
- A DOI is stored without its ``https://doi.org/`` prefix, so the column holds
one spelling whichever way it was pasted.
## MCP server ## MCP server
@@ -513,6 +576,7 @@ paper-doc/
│ ├── alembic/ # migration environment and revisions │ ├── alembic/ # migration environment and revisions
│ ├── scripts/seed.py # idempotent seed for the title library + templates │ ├── scripts/seed.py # idempotent seed for the title library + templates
│ ├── scripts/smoke_papers.py # end-to-end check of the writing loop │ ├── scripts/smoke_papers.py # end-to-end check of the writing loop
│ ├── scripts/smoke_references.py # end-to-end check of the reference library
│ ├── scripts/mcp_server.py # MCP entry point (stdio / http) │ ├── scripts/mcp_server.py # MCP entry point (stdio / http)
│ ├── scripts/smoke_mcp.py # the same loop, spoken over MCP by a real client │ ├── scripts/smoke_mcp.py # the same loop, spoken over MCP by a real client
│ ├── alembic.ini │ ├── alembic.ini
@@ -592,6 +656,18 @@ cd backend
.venv/bin/python scripts/seed.py --reset # empty the tables first .venv/bin/python scripts/seed.py --reset # empty the tables first
``` ```
**Checking the reference library end to end**
`scripts/smoke_references.py` adds literature, finds it again by author, type,
year and whether it has been cited, edits it, cites it from a sentence, looks
the citation up from the reference side, and checks that a cited entry cannot be
deleted. It cleans up after itself:
```bash
cd backend
.venv/bin/python scripts/smoke_references.py
```
**Checking the paper feature end to end** **Checking the paper feature end to end**
`scripts/smoke_papers.py` walks the whole writing loop against a running API — `scripts/smoke_papers.py` walks the whole writing loop against a running API —