Commit Graph

12 Commits

Author SHA1 Message Date
govin 15955f552d backend: give the reference library the MCP tools it was missing, and fix the one that broke
The reference library landed with a full REST API and no MCP surface at all:
an agent could cite `reference_id: 42` — the citation is validated now — but
could not look up what 42 was, create it, or find out whether the library
already had it. That is the citation half of "generate the paper, then write it
in" missing, and it is the half an agent is worst at doing by hand.

Twelve tools, mirroring the REST surface plus one addition each way:

- `reference_*` — search, read, create, update, delete, plus
  `reference_import`, which takes a whole pasted bibliography in one call and
  reports what it skipped (an existing DOI or an identical title) and what it
  could not read, one bad entry at a time rather than losing the other nineteen;
  and `reference_citations`, which answers "what is this cited for" before a
  rename or a delete. `reference_list`'s keyword search across title, container,
  DOI and author names is the de-duplication path that `reference_import` uses.
- `reference_tag_*` — the five CRUD tools. Deleting a tag drops the labelling
  and leaves the literature, which is the opposite of what the field library
  needs and the same choice the REST route makes.

Two things had to be fixed to get here, both found by testing rather than
reading:

`template_list` was crashing. The rename that turned a heading into a title and
its use into a placement changed `TemplateListItem.field_count` to
`placement_count` and left the tool reading the old attribute, so the call died
with `Error executing tool template_list` — unreadable, and live in the
cluster. The suite passed 47 checks while this was broken because it never
called `template_list`: the uncovered tools were exactly the read-only ones, so
the write paths were all green and the browse path was not. The suite now
covers all 41 tools — 76 checks — and step 5 asserts the shape each read tool
returns, not merely that it returned.

`DOMAIN_REFUSALS` in the tool registry held one class, `UnknownReference`. The
tag rules arrived with the tag tools and raise two more, so 「标签不存在:9」 and
「标签「必读」已存在」 would have reached a model as `Error executing tool …`.
The tuple is now the list of every refusal a tool is expected to translate, and
a new one belongs in it on the day it is written.

`reference_update` takes its changes as one nested object rather than 38 keyword
parameters: a nested model keeps `model_fields_set`, so an omitted field stays
absent while an explicit null clears the column. Flattened, the two are
indistinguishable — every omitted field arrives as None — which is the same
problem `paper_update` solves by rebuilding its payload from what it was given.

Verified against the deployed service, not a local process: 76 checks pass over
the NodePort, then the same 76 over stdio and against a local HTTP server, and
the REST suite still passes its 55. The image is rebuilt as v4 and the
Deployment follows it, because `imagePullPolicy: IfNotPresent` means re-pushing
a tag the node already holds changes nothing.
2026-09-22 18:30:24 +08:00
govin 6020310b3f backend: tag the literature, in a table of its own
A tag is the writer's filing of a row — 「综述」, 「必读」 — and it is a
different fact from `reference.keywords`, which are the 关键词 printed on
the article. A tag never prints, one document carries several, and the same
label is on many documents, so it gets two tables rather than a column:

  reference_tag       name (unique), colour, note
  reference_tag_link  reference_id + tag_id — the pair is the primary key

The link is a bare link, which is why it is `_link` and not `_placement`:
there is no `sort` to own, so the pair *is* the row and the same label
cannot be applied twice.

A tag is made first and applied second: `tag_ids`, never tag names, so a
typo cannot invent a label — an unknown id is a 400 naming it, and nothing
is written. Deleting a tag is deliberately *not* guarded the way deleting a
cited reference is: both foreign keys cascade, so the labels go and the
literature stays.

`GET /references?tag=1&tag=2` narrows rather than widens — the rows carrying
both labels — because that is what a second filter reads as.

The smoke test now walks the whole flow in the order the feature intends:
labels first, then tagged references, then a rename that follows every row.
2026-09-22 17:55:15 +08:00
govin d7bb777264 backend: add the reference library (参考文献)
One table per document the writer has read, and one for its contributors:

    reference          id, type, title, subtitle, title_translated,
                       container_title, volume, issue, pages, publisher,
                       publisher_place, event_*, publication_year/month/day,
                       accessed_date, doi, isbn, issn, url, patent_*,
                       standard_number, report_number, arxiv_id, abstract,
                       keywords, note, citation_key, language
    reference_author   reference_id, sort, name, family_name, given_name, role

The field set is the intersection of the formats a paper has to survive, taken
from the CSL JSON schema (103 fields, 45 types — what Zotero, Mendeley and
Pandoc exchange) against GB/T 7714-2015, BibTeX and RIS. The mapping is on
`app.models.reference` rather than in a commit message, because the next person
to add a column needs it more than this one does. Three decisions worth naming:

* **Partial dates.** Year, month and day are separate nullable integers, which
  is CSL's `issued`: a book has a year, a newspaper article a day, a patent its
  公告日期, and nothing invents the rest.
* **Contributors are rows, not a column.** A citation prints an ordered list
  with 编 / 译 marks, and styles rewrite names (「张三」 / "Zhang, S." /
  "S. Zhang"), so the printed form and its two halves are stored separately —
  display reads `name`, a reformatting style reads the halves, and an
  institutional author has neither.
* **`volume` / `issue` / `pages` are strings.** Journals print `12(Suppl.)`,
  `S1` and `e12345`, and a citation reproduces what is printed.

The library is global: the same article is cited from several papers, so the
entry cannot belong to one of them. The citation stays on the paper side in
`paper_sentence_reference`, which now carries a real foreign key to `reference`
(`ON DELETE RESTRICT`) — the column had been a loose integer waiting for this
table. A sentence may quote several documents and a document may be quoted by
many sentences, which is why the relation is a table and not a column on
`paper_sentence`.

Refusals are raised once, in the data layer, as `UnknownReference`, and
translated where each caller speaks: `400` with the offending ids for REST (one
exception handler, no route carries the check), a readable tool error for MCP
(one wrapper in the tool registry). Neither entry point can skip it, so the
foreign key never has to be the thing that complains.

API: `/references` CRUD plus batch delete, and `/references/{id}/citations` for
the reverse lookup — "where have I used this article?" — which is what tells a
writer whether an entry is safe to delete.

Verified: 29 new checks in `scripts/smoke_references.py` (partial dates, DOI
normalisation, author order and roles, search by author/type/year/cited, the
citation round trip, 400 on a phantom id, 409 on a cited entry), the paper
suite's 45 and the MCP suite's 47 still pass. The MCP suite now cites a real
entry instead of a phantom id, which is what an agent has to do now.
2026-09-22 17:13:32 +08:00
govin 199e328b48 refactor: call a heading a title, and its use a placement
`template_field_library` and `template_field` were one word apart and meant
opposite kinds of thing — a catalogue of rows versus one row — and neither name
said what a row actually is. Every row of the library is a **heading**: a piece
of text like 「1. Introduction」 with a level and typography, which is what the
reader sees at the top of a paragraph; and every row of the join table is that
heading *placed* in a template, `sort` included.

    template_field_library  -> template_title
    template_field          -> template_title_placement
    field_id (the column)   -> title_id
    TemplateFieldLibrary    -> TemplateTitle
    TemplateField           -> TemplateTitlePlacement

The join table keeps its own name rather than becoming `_link`, because it is
not a bare link: it carries `sort`, the only place display order is stored, and
it deliberately allows the same title twice in one template. `_link` would
describe two foreign keys and hide the column the ordering rule rests on.

Carried through every level, because a half-renamed layer is how the confusing
pair got there in the first place: HTTP paths (`/template-field-library` ->
`/template-titles`), the template payload (`fields`/`field_id` ->
`placements`/`title_id`, and `field_count` -> `placement_count`), the Pydantic
schemas, the CRUD modules, the seed script, the frontend API module and types,
and `docs/OVERVIEW.md`, which now states the naming rules outright.

Two deliberate exceptions:

* `paper_template_filed_sort` is untouched — it is a column of the paper module
  and is spelled as the feature was specified.
* The MCP tools keep their agent-facing names (`field_list`, `field_ids`, and
  the `fields`/`field_id` keys in their responses). Agents already call them by
  those names; the storage names they read underneath have changed.

The migration renames in place (data preserved: 4 templates, 45 titles, 63
placements) and is written to tolerate a database where part of it already ran,
which is not hypothetical — the first attempt renamed the tables and then failed
on the column, because Alembic's MySQL dialect wants an explicit type for
`alter_column` and will not rename a column without one. TiDB carries the
foreign key along with both the renamed table and the renamed column, rehearsed
against the cluster first, and it stays enforced.

Verified: seed re-runs clean, the paper smoke test's 45 checks and the MCP
smoke test's 47 pass, `vue-tsc` and the build are clean, and the UI was driven
in a headless browser through the template list, its preview drawer, the title
library page, the template form (which now posts `placements` with `title_id`)
and the paper document with no console errors.
2026-09-22 16:55:02 +08:00
govin 7e7820f823 deploy: run the MCP server in k3s under the paper-doc namespace
The server has to be reachable by the Harness, Claude Code and Codex, and all
three of those are clients of an address rather than of a source tree — a
stdio child process only serves whichever machine spawned it. So it is packaged
and deployed: namespace `paper-doc`, one Deployment, one NodePort Service at
`http://192.168.1.88:30931/mcp` behind a bearer token, talking to the
in-cluster TiDB Service rather than the node's NodePort.

The namespace is `paper-doc` and not `paper_doc`, which is not a preference:
a namespace name is an RFC 1123 label, underscores are outside that alphabet,
and the API server rejects it outright. The database keeps its underscore.

Two routes were added to the server for the deployment, and they are the only
unauthenticated ones. `/healthz` answers "is this process alive" without
touching anything, `/readyz` answers "can it serve" by running `SELECT 1`. The
split is the point: a failing readiness probe takes the pod out of the
Service's endpoints and puts it back when TiDB returns, while a failing
liveness probe kills a process that was working correctly.

There is no container runtime on this host — k3s runs containerd and neither
`ctr` nor `crictl` builds from a Dockerfile — so `deploy/build-mcp-image.sh`
builds inside a throwaway privileged docker-in-docker pod, mounting the working
tree through a hostPath so the build context is the tree you are looking at,
uncommitted edits included. It reads the registry credentials out of the
cluster's own pull secret, so no password is in the script or in a shell
history. One Dockerfile serves both entry points, because the API and the MCP
server share the domain layer and differ by one argv.

Verified against the deployed Service, not against a local process: 47 MCP
checks pass over the NodePort, and the Node MCP SDK — the one the Harness's
mcp-client plugin uses — lists all 29 tools and calls them. The same 47 pass in
stdio and against a local HTTP server, and the REST suite still passes its 45.
2026-09-21 00:08:17 +08:00
govin 2aff867641 backend: serve the writing domain to agents as an MCP server
The paper is written by a model now. A browser is the wrong client for that:
the work is "generate the paper, then put it in", and doing it through a form
means a person retyping what a model already produced. So the same domain is
served over the Model Context Protocol, which Claude Code, Codex and the
DeepSeek Harness all speak.

It is a third front door, not a second implementation. Every tool is three
lines around an `app.crud` call and validates through `app.schemas`, exactly
as the REST routes do, so a rule fixed in the CRUD layer is fixed on both
surfaces and a paper written by an agent is indistinguishable from one written
by hand. What `app/mcp/` adds is only what a model needs and a browser does
not:

- 29 tools, prefixed `paper_` `paragraph_` `sentence_` `template_` `field_`,
  because a model picks a tool out of a list by name rather than by reading 29
  descriptions;
- results as compact `None`-free JSON, since a tool result is paid for in
  context tokens and `PaperRead.model_dump()` carries four counts and two
  timestamps into every list row;
- paragraphs addressed by **heading** as well as by position. Storage is
  correct as it stands — a sentence remembers the position it sits at, which is
  what makes a template switch non-destructive — but nobody writing
  "1. Introduction" knows the template places it at `sort = 20`. The server
  translates, and refuses with the real heading list when it cannot, so a model
  that guessed wrong corrects itself in one retry;
- `paper_write` and `paper_write_text`: one intention, one call. The latter
  finds its own sections from Markdown headings or from lines that name a
  template heading, and reports every heading it could not place instead of
  writing half a paper;
- `sentence_search` across papers, for consistency rather than retrieval — a
  paper that says 洪水损失 should not be joined by one that says GUL;
- `paper_delete` refuses once, naming what would go with it. A cascading delete
  has no undo in a tool call.

Two transports, one build. `stdio` is what a client spawns — so nothing in the
process may print to stdout, and diagnostics go to stderr. `streamable-http` is
what a client on another machine connects to, optionally behind a bearer token;
binding a non-loopback address disables the SDK's DNS-rebinding allow-list,
because a LAN client sends whatever Host it knows the server by.

Tools register with `structured_output=False` on purpose: inferred from a
`-> str` annotation the SDK publishes a `{"result": ...}` envelope and sends
the JSON twice, once as `structuredContent` and once as text, and clients that
read only one of the two then disagree about what came back.

`scripts/smoke_mcp.py` drives the whole loop through a real MCP client — the
child process and JSON-RPC over stdin/stdout a client actually uses — and runs
unchanged against a running HTTP server via `--url`. 47 checks pass on both
transports; the REST suite still passes its 45.
2026-09-19 00:01:43 +08:00
govin e582198cc8 backend: decide the seam between sentences instead of gluing them
A paragraph is its sentences concatenated. Nothing was put between them, which
is right for Chinese — 「。」 already separates — and wrong for English, where
`Adaptive capacity rises.` followed by `Relocation follows.` printed as
`...rises.Relocation...`: a sentence boundary the reader cannot see. CJK output
hid it, so it would have surfaced as an English bug later rather than now.

`sentence_separator` now returns "" or a single space per seam, and the document
carries it as `SentenceRead.separator_before`. The rule is about the seam, not
the language: a space goes in unless both sides are CJK. Mixed seams take the
space. Empty sentences take none. It is derived on every read and never stored,
so it cannot drift from the text, and the client prints `separator_before +
content` and adds no spacing of its own.

No splitting was added anywhere, and none exists: a sentence is one line in the
editor and nothing parses it. The single split this project has ever performed
was the one-time move of the old abstract column, which cut after 「。」 only —
conservative on purpose, since an English abstract is better left in one row
than cut at the first `et al.`.

Five smoke checks cover the seam (Chinese, English, the assembled paragraph,
and that no separator is stored inside the content). 45 checks pass.
2026-09-18 19:01:31 +08:00
govin 45926af966 refactor: make the abstract a paragraph instead of a paper field
A template's abstract paragraph (`0 Abstract`) *is* the paper's abstract: it has
a position in the document, the heading and typography the template gives it,
and the same sentence-by-sentence editing as everything else. `paper.abstract`
was a second home for that text — one the document never reads, so a paper
could show two different abstracts and the column could drift from the body.

The column is gone, from the table, the model, the schemas, the API payloads,
the paper form, the list subtitle and the paper page. Nothing else changed.

The text already written into it is not gone. Revision a83f5c21d7b6 writes each
stored abstract into the paper's body first — one sentence per 。 at the
paragraph carrying the abstract heading, appended after anything already there
rather than replacing it. A template without such a heading keeps the text too,
one position above its first paragraph, where the document renders it under
未设定. Verified against the one paper that had an abstract: 450 characters in,
450 out, identical including `|J| ≤ α · I⁻ᵝ`, split into six sentences in the
`0 Abstract` paragraph, still with its own edit button.

The smoke test no longer assumes an empty database: it records the paper total
before it starts and compares against that, so it can run on a real one.
2026-09-18 18:55:10 +08:00
govin a5f884f440 refactor: prefix every table with its module
Two tables were named after the concept they came from rather than the module
they belong to, so the schema read as if the template tables were part of the
paper module. Renamed (data preserved, `RENAME TABLE` moves rows in place):

    paper_template  -> template                 the 模板 module
    section_field   -> template_field_library   the 字段库 the 模板 module owns

The paper tables and `template_field` already followed the rule. The rename
carries through everything that named a module:

  models   Template, TemplateField, TemplateFieldLibrary
  schemas  Template*, TemplateFieldLibrary*
  crud     app/crud/template.py, app/crud/template_field_library.py
  API      /template-field-library (was /section-fields); handlers are now
           named after library entries, which removes the ambiguity with
           TemplateField — a placement, a different thing entirely
  client   src/api/templateFieldLibrary.ts

`paper_template_filed_sort` is deliberately untouched: it is a column of the
paper module, spelled as the feature was specified.

TiDB v8.5 with tidb_enable_foreign_key on — as this cluster runs — enforces
foreign keys rather than ignoring them, so the docs' "TiDB does not enforce
foreign keys" was wrong. Corrected, with what actually follows from it: the
rename was rehearsed (RENAME TABLE carries a referencing constraint along), the
API keeps checking first so a violation names the row instead of surfacing a
driver error, and the ORM cascades stay so behaviour does not depend on a
cluster setting.

Revision f27a1c6d9e04 verified both ways; 40 smoke checks, type-check and build
all pass.
2026-09-18 17:48:11 +08:00
govin 06d7e922bd backend: add the paper authoring schema, API, and smoke test
A paper is a document written against a template. Its structure is never
copied into it: `paper` points at a template and the outline is read live on
every render, so switching `template_id` re-shapes the whole document in one
write.

Sentences are addressed by *position* rather than by a template row id:
`paper_sentence.paper_template_filed_sort` holds the sort of the placement the
sentence belongs to, and `sort` holds its place inside that paragraph. That
indirection is what makes a template switch non-destructive — a sentence that
remembers "position 7" lands on whatever the new template puts at position 7 —
and it is why a paragraph is any position either the template or the content
mentions: the structure survives with no content, and content whose position
the template does not define is still rendered, in order, under 未设定.

Citations are a table rather than a column, since one sentence may quote
several references. `quote` is required — a citation that does not say what it
quotes is refused with 422 — while `reference_id` is a plain nullable integer
with no foreign key, because the reference library does not exist yet.

Deleting a template a paper is written against is refused with 409 and a count,
matching how the field library refuses to drop a field still in use.

scripts/smoke_papers.py walks the whole loop — create, empty structure, write a
paragraph with citations, switch templates, keep unmatched positions, move a
paragraph, delete — in 40 checks, and cleans up after itself.
2026-09-18 17:29:12 +08:00
govin 13e5fc2cc1 backend: add section-field and paper-template schema, API, and seed data
Three tables behind the template configuration feature:

  section_field   the reusable heading library (name, level, font size,
                  colour)
  paper_template  a named outline (name, abstract)
  template_field  the join, and the only home of display order (sort)

The field library is deliberately flat. A `parent_id` would tie a level-2
heading to exactly one level-1 heading, and the point of the feature is that
a field such as "Background" can sit under both "1. Introduction" and
"2. Related Work" — and in any number of templates, at a different position
in each. Hierarchy is expressed only by `level`, which is a rendering hint.

Display order lives entirely in `template_field.sort`. The order fields were
picked in is never stored, so selecting fields out of order and assigning
sorts renders in sort order. Two consequences are intentional and documented
on the model: repeats are allowed (no unique constraint on template+field)
and ties are legal (broken by insertion order, so the ordering is total).

Templates reference library fields rather than copying them, so renaming or
restyling a field updates every template that places it.

TiDB parses FOREIGN KEY and then ignores it, so the constraints are declared
for documentation and the integrity is enforced in the application layer:
deleting a field still in use is refused with the field names, creating a
template against a missing field is refused, and deleting a template removes
its join rows through the ORM's delete-orphan cascade.

Also normalises font_color to #RRGGBB on write (accepting rgb() and
shorthand) and emits font_size as a JSON number rather than pydantic's
default Decimal string.

scripts/seed.py is idempotent and fills the library with a standard academic
outline plus three starter templates.
2026-09-18 15:56:47 +08:00
govin 5128e1551c feat: scaffold FastAPI backend and Vue 3 frontend
Backend (backend/):
- FastAPI app with layered app/{api,core,crud,db,models,schemas} structure
- TiDB connection via SQLAlchemy. The URL is built with URL.create rather
  than string formatting: the password contains '@', which would otherwise be
  parsed as the user/host separator and silently truncate the credential.
- Alembic environment wired to Base.metadata and the app settings, so
  backend/.env stays the single source of truth for credentials
- /api/health endpoint reporting database reachability
- models/ and crud/ are intentionally empty: no model classes are defined and
  no tables are created, at import time or otherwise

Frontend (frontend/):
- Vue 3 + Vite + TypeScript scaffold (create-vue, --bare)
- axios instance with a normalized error shape
- vue-router with a home route and a catch-all 404
- pinia + pinia-plugin-persistedstate; the app store persists selected keys
- Element Plus with its icons registered globally
- dev proxy forwards /api to the FastAPI service on port 8000

The database paper_doc was created in TiDB out of band; no table exists yet.
2026-09-17 12:51:37 +08:00