backend: decide the seam between sentences instead of gluing them

A paragraph is its sentences concatenated. Nothing was put between them, which
is right for Chinese — 「。」 already separates — and wrong for English, where
`Adaptive capacity rises.` followed by `Relocation follows.` printed as
`...rises.Relocation...`: a sentence boundary the reader cannot see. CJK output
hid it, so it would have surfaced as an English bug later rather than now.

`sentence_separator` now returns "" or a single space per seam, and the document
carries it as `SentenceRead.separator_before`. The rule is about the seam, not
the language: a space goes in unless both sides are CJK. Mixed seams take the
space. Empty sentences take none. It is derived on every read and never stored,
so it cannot drift from the text, and the client prints `separator_before +
content` and adds no spacing of its own.

No splitting was added anywhere, and none exists: a sentence is one line in the
editor and nothing parses it. The single split this project has ever performed
was the one-time move of the old abstract column, which cut after 「。」 only —
conservative on purpose, since an English abstract is better left in one row
than cut at the first `et al.`.

Five smoke checks cover the seam (Chinese, English, the assembled paragraph,
and that no separator is stored inside the content). 45 checks pass.
This commit is contained in:
2026-09-18 19:01:31 +08:00
parent 898f519773
commit e582198cc8
6 changed files with 181 additions and 13 deletions
+5
View File
@@ -167,6 +167,11 @@ class SentenceRead(BaseModel):
sort: int
content: str
citations: list[CitationRead]
#: Text to print before this sentence when the paragraph is put back
#: together — ``""`` or a single space, never anything else. Derived on
#: read by :func:`app.crud.paper.sentence_separator`, never stored, because
#: a separator that is stored is a separator that can go stale.
separator_before: str = ""
created_at: datetime
updated_at: datetime