Developer Guide¶

Architecture¶

Frontend-Request
    │
    ├─ POST /nt-lingua/transform ─→ TransformApiMiddleware
    │        │
    │        ├─ isLanguageTarget(target) = true  ─→ TextTransformationService::translateStrings()
    │        └─ isLanguageTarget(target) = false ─→ TextTransformationService::transformField()
    │
    └─ GET page request (cookie/param)
             └─ TransformProcessor (DataProcessing) ─→ transformField() [server-side fallback]

TextTransformationService
    ├─ einfache / leichte  ─→ AiProviderInterface (nt_ai)    + Prompts
    └─ language codes       ─→ TranslatorInterface (LLM/DeepL) + Glossary

TextCacheRepository
    └─ tx_ntlingua_text_cache (UNIQUE: content_hash + target)

Two coupling points:

Frontend → Server: POST /nt-lingua/transform handled by TransformApiMiddleware
Server → AI/Translation: TranslatorInterface (LLM or DeepL) + AiProviderInterface (nt-ai)

nt_lingua does not reference any concrete AI provider — the only external coupling is Netthinks\NtAi\Provider\AiProviderInterface.

Response format and monitoring¶

POST /nt-lingua/transform with a language target responds:

{
  "target": "fr",
  "translations": { "source text": "translation" },
  "untranslated": 0,
  "degraded": false
}

Field / header	Meaning
`translations`	Source text → translation. A segment that could not be translated carries its source text, so the page stays complete.
`untranslated`	Number of segments without a translation
`degraded`	`true` as soon as at least one segment failed
`X-NtLingua-Untranslated`	The same count as a header — for checks without parsing JSON

Status codes: A partial failure responds 200 (the translated part is still useful), a complete failure responds 503. This makes no difference to the rendering — the client script does not evaluate the status, it applies whatever it receives and otherwise leaves the page in the source language.

The budget and load limits add their own status codes: 429 when the quota is used up, 503 with Retry-After when all concurrent provider slots are taken. Monitoring should therefore read untranslated or the header rather than the status alone.

Why these signals exist: without them an outage is invisible from the outside. The response then carries the source texts and looks like a complete translation.

Failed segments are not cached — the next call retries them. Every failure also adds an entry to tx_ntlingua_log whose error field states the count. That count refers to the provider call, i.e. only the segments not already cached; the request-level number is untranslated.

Data model¶

tx_ntlingua_text_cache¶

Column	Type	Description
`content_hash`	varchar(64)	`sha256(normalize(source) \\| target \\| model)`
`target`	varchar(20)	`einfache`, `leichte` or language code (e.g. `en`)
`source_lang`	varchar(10)	Source language (default: `de`)
`source_text`	mediumtext	Original text
`result_text`	mediumtext	Transformed/translated text
`provider`	varchar(30)	AI provider or `deepl`
`model`	varchar(80)	Model used
`tokens_used`	int	Tokens consumed
`readability`	decimal(5,2)	Avg. words per sentence (lower = simpler)
`source_uid`	int	UID of the source record
`source_table`	varchar(80)	Table of the source record
`source_field`	varchar(80)	Field of the source record
`crdate` / `tstamp`	int	Unix timestamps

Unique key: (content_hash, target) — Einfache, Leichte and all language codes share the table without collision.

Cache invalidation: AfterDatabaseOperationsEvent → InvalidateTextCache deletes entries for the affected record when header or bodytext is saved.

tx_ntlingua_glossary¶

Column	Description
`source_term`	Source term (normalised before comparison)
`target_lang`	Target language (ISO code) or `*` for all languages
`target_term`	Fixed translation (leave empty for `ignore` mode)
`mode`	`fixed` (force) or `ignore` (leave unchanged)

Glossary entries are evaluated before the cache and the translation provider. target_lang='*' applies language-independently to all DOM translations.

tx_ntlingua_ratelimit¶

Fixed-window counters for the public endpoints (since 1.1.0). One row per dimension and window (bucket), the consumed units (used) and the purge time (expires). Rows are only changed through a conditional UPDATE … SET used = used + n WHERE used <= max - n, so parallel requests cannot exceed a limit.

tt_content / pages extension fields¶

Field	Description
`tx_ntlingua_mt`	`1` = machine-translated (readonly in backend)
`tx_ntlingua_srchash`	Hash of source fields at overlay creation time — enables delta detection

CLI reference¶

ntlingua:warmup¶

vendor/bin/typo3 ntlingua:warmup [--target=<target>] [--limit=<n>]

Option	Default	Description
`--target`	`einfache`	`einfache`, `leichte`, or an ISO language code
`--limit`	`0`	Max. records to process (0 = all)

Processes all tt_content records with sys_language_uid=0, deleted=0, hidden=0. Cache hits are skipped — the command is idempotent and safe to run as a nightly scheduler task.

# Pre-fill Einfache Sprache for all content elements:
vendor/bin/typo3 ntlingua:warmup --target=einfache

# Pre-fill English translation, at most 500 records:
vendor/bin/typo3 ntlingua:warmup --target=en --limit=500

ntlingua:cache-purge¶

vendor/bin/typo3 ntlingua:cache-purge --target=uk,sr [--model=gpt-4o-mini] [--dry-run]

Removes cached translations of the given target languages (since 1.4.3), so the next ntlingua:warmup-dom translates them again. For results that are wrong although the call succeeded — the cache key knows text, target and model, not the prompt. --target is required; there is deliberately no "all languages".

ntlingua:overlays¶

vendor/bin/typo3 ntlingua:overlays --language=<uid> --code=<iso>

Option	Description
`--language`	`sys_language_uid` of the target language (must exist in TYPO3 site config)
`--code`	ISO language code (e.g. `en`, `fr`)

Creates connected-mode l10n overlays. Only changed records are processed on repeated runs (delta detection via tx_ntlingua_srchash).

Translated fields:

Table	Fields
`pages`	`title`, `nav_title`, `subtitle`, `seo_title`, `description`
`tt_content`	`header`, `subheader`, `bodytext`

Warning

Run ntlingua:overlays in the live workspace, not in draft workspaces.

JavaScript API¶

// Switch language or mode:
NtLingua.transform('fr', buttonElement);   // DOM translation
NtLingua.transform('einfache', btn);       // Plain language
NtLingua.transform('de');                  // Reset to source language

// Query current state:
NtLingua.getCurrent();   // e.g. 'de', 'fr', 'einfache'

Custom events¶

document.addEventListener('NtLingua:ready', (e) => {
    console.log('Initialised, active target:', e.detail.target);
});
document.addEventListener('NtLingua:done', (e) => {
    console.log('Switch complete:', e.detail.target);
});

Adding a custom translator¶

Implement TranslatorInterface and register it as an alias in Services.yaml:

namespace MyVendor\MyExt\Translator;

use Netthinks\NtLingua\Translator\TranslatorInterface;

final class MyTranslator implements TranslatorInterface
{
    public function translateBatch(array $texts, string $sourceLang, string $targetLang): array { … }
    public function getName(): string  { return 'my-translator'; }
    public function getModel(): string { return 'my-model'; }
}

# Services.yaml of your extension:
Netthinks\NtLingua\Translator\TranslatorInterface:
  alias: MyVendor\MyExt\Translator\MyTranslator

Fluid wrappers (required for DOM field swap)¶

Content fields must be wrapped with data-ntlingua-* attributes in FSC template overrides:

<!-- bodytext (e.g. in an EXT:fluid_styled_content override): -->
<div data-ntlingua-uid="{data.uid}" data-ntlingua-field="bodytext">
    <f:format.html>{data.bodytext}</f:format.html>
</div>

<!-- header: -->
<div data-ntlingua-uid="{data.uid}" data-ntlingua-field="header">
    {data.header}
</div>

Place Fluid overrides under packages/netthinks/Resources/Private/Extensions/fluid_styled_content/.

Security of the public endpoints¶

ContentAccessGuard decides with the frontend's own rules (PageRepository, RootlineUtility, RecordAccessVoter) whether a content element or page may be used for an anonymous request: hidden, start/stop time, access groups, "extend to subpages", site and language. Nothing is loaded into a provider before that check.
TextTransformationService::transformField() returns sanitized HTML for HTML sources and plain text for plain sources — independent of the provider answer; cache hits are sanitized again on output. Consumers that write into HTML must escape plain text (TransformApiMiddleware sends format, EinfacheSpracheMiddleware escapes before restoring the page's own tags).
Public callers pass a ProviderBudget (from PublicUsageLimits::budgetFor()); it is reserved right before every provider call, including the LLM fallback. Backend/CLI code passes none, and translators then behave exactly as without budgets.
Translators that may send more than once per batch implement BudgetAwareTranslatorInterface and reserve themselves before each send (LlmTranslator: every chunk, every resend of an empty chunk and, since 1.4.2, every half of a chunk whose answer did not fit — wrong length, no JSON array, or an element that is not text; chunks are halved until the answer fits or a single string is left, which then stays null; when the budget refuses, splitting stops and what arrived is kept; DeepLTranslator: every request, including resends after 429). Contract: BudgetExceededException means the batch produced nothing; a refusal after something was sent returns the batch with untranslated entries left as source text, which TextTransformationService neither returns nor caches. A plain TranslatorInterface implementation is treated as a single send and reserved once up front.
RateLimitedProviderBudget also takes one of maxConcurrentProviderRequests slots (ConcurrencySlots, rows slot_<n> in tx_ntlingua_ratelimit, taken by a conditional UPDATE or the creating INSERT) on its first reservation, and holds it until release(), its destruction or the lease timeout. A full house raises ConcurrencyLimitExceededException (a BudgetExceededException); the API answers 503. Race test against MariaDB: Tests/Concurrency/concurrency-slots-race.php.
Accepted limitation (not a fix): DOM translation accepts rendered segments that are not records, so arbitrary text can be translated within the budgets unless domTranslationCacheOnly is enabled; a caller can also use up the global budget for everyone until the window ends.

Tests¶

composer install
typo3DatabaseDriver=pdo_sqlite vendor/bin/phpunit -c phpunit.xml.dist

Functional tests run on SQLite with in-process fake providers — no paid API is ever contacted. Tests/Concurrency/rate-limiter-race.php and Tests/Concurrency/concurrency-slots-race.php check the counters and the concurrency slots with real parallel processes against MariaDB/MySQL (see the file headers). Tests/JavaScript/dom-switch.cjs checks the DOM translation in the browser (Playwright, since 1.4.4): switching between two foreign languages in cache-only mode on network errors and with a slider that re-appends its slides — NODE_PATH=<path>/node_modules node Tests/JavaScript/dom-switch.cjs.

Accessibility¶

<html lang> and dir are updated on every switch (WCAG 3.1.1 / 3.1.2)
Language names rendered as autonyms with per-button lang attributes
The dropdown carries data-nt-notranslate — language names are not translated when the page itself is translated
role="status" / aria-live="polite" for error messages in the navbar area
aria-live="polite" on the disclaimer banner
aria-busy="true" on <main> during translation (overlay spinner)
aria-pressed on buttons for active state; native languages as links with aria-current
Native languages as links (semantically correct, usable without JS, no WCAG 3.2.2 issue)
DOM languages and Plain Language as buttons (no page load, pure client-side behaviour)
data-ntlingua-native on native links → localStorage cleanup before navigation
Fully keyboard-navigable, visible focus; widget closes on Escape

Known issues¶

PHP: numeric string keys in arrays¶

PHP automatically converts array keys that look like integers ("1999", "42") to int. In TextTransformationService::translateStrings() this can cause $orig to be int in the foreach ($missing as $orig => $_) loop, even though normalize(string $s) expects a string. Fix: add $orig = (string)$orig; at the top of the loop.

DeepL: `source_lang` does not accept locale variants¶

DeepL's source_lang parameter only accepts base codes (EN, DE), not locale variants (EN-US, DE-AT). The <html lang> attribute may contain a locale variant (e.g. en-US when locale: en-US is set in the TYPO3 site configuration). DeepLTranslator truncates the source code with explode('-', $sourceLang)[0].

Chrome Autotranslate blocks the TreeWalker¶

Chrome's built-in translator sets class="notranslate" on the <html> element. The ignored() function explicitly stops at document.documentElement and does not check the <html> element itself — otherwise the entire page content would be treated as excluded and the TreeWalker would return an empty text pool.

Two switcher instances (mobile + desktop)¶

The partial is rendered twice (mobile and desktop, controlled by CSS). setToggleLabel() uses querySelectorAll('.nt-lingua-active-label') to keep both toggle buttons in sync.