Recognizers

Supported recognizers:

TRBA
PPOCRRec
TrOCR
MajorityVotingRecognizer
MedoidRecognizer

from manuscript.recognizers import TRBA, PPOCRRec, TrOCR
from manuscript.utils import create_page_from_image

recognizer = TRBA(weights="trba_lite_g2")
page, image = create_page_from_image("word.png", return_image=True)
result = recognizer.predict(page, image=image)
print(result.get_text())

TRBA

class manuscript.recognizers.TRBA(weights=None, config=None, charset=None, device=None, force_download=False, rotate_threshold=1.5, region_preparer='bbox', region_preparer_options=None, min_text_size=5, batch_size=128, **kwargs)[source]

Bases: BaseRecognizer

Initialize TRBA text recognition model with ONNX Runtime.

Parameters:

weights (str or Path, optional) –
Path or identifier for ONNX model weights. Supports:
- Local file path: "path/to/model.onnx"
- HTTP/HTTPS URL: "https://example.com/model.onnx"
- GitHub release: "github://owner/repo/tag/file.onnx"
- Google Drive: "gdrive:FILE_ID"
- Preset name: "trba_lite_g1" or "trba_base_g1" (from the model registry)
- None: auto-downloads default preset (trba_lite_g1)
config (str or Path, optional) – Path or identifier for model configuration JSON. Same URL schemes as weights. Required for local weights; registry presets supply their own config artifact. The config must include recognition_levels, e.g. ["word"]. Pass recognition_levels=["word"] to override this selection explicitly.
charset (str or Path, optional) – Path or identifier for character set file. Required for local weights; registry presets supply their own charset artifact.
device ({"cuda", "coreml", "cpu"}, optional) –
Compute device. If None, selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. For GPU/CoreML acceleration:
- CUDA (NVIDIA): pip install onnxruntime-gpu
- CoreML (Apple Silicon): included in standard macOS ONNX Runtime wheels; select device="coreml" explicitly.
Default is None (automatic selection).
rotate_threshold (float or None, optional) – Aspect-ratio threshold for rotating vertical text-span crops before recognition. If height > width * rotate_threshold, crop is rotated 90 degrees clockwise. Set to 0 or None to disable. Default is 1.5.
region_preparer ({"bbox", "polygon_mask", "quad_warp"} or callable, optional) – Strategy used to convert Page polygons into recognition crops. "bbox" extracts axis-aligned bounding boxes for arbitrary polygons. "polygon_mask" masks pixels outside the polygon inside a tight crop and also supports arbitrary polygons. "quad_warp" rectifies only 4-point polygons with a perspective transform before recognition. A custom callable may also be provided and should return a list of prepared text regions. Default is "bbox".
region_preparer_options (dict or None, optional) – Optional configuration for built-in region preparers. Defaults to None. Typical options are pad for "bbox" and "polygon_mask", or output_size=(width, height) for "quad_warp". Non-quad polygons passed to "quad_warp" fall back to bbox crops by default. For "bbox", padding_x, padding_y, inset_x and inset_y are nonnegative fractions of the original bbox size per side (default 0). Insets must be below 0.5. Padding and inset on the same axis cannot be combined; relative margins cannot be combined with nonzero pixel pad. For example, {"inset_y": 0.1} removes 10% from the top and bottom before orientation compensation, without changing the Page geometry. Other presets reject these options.
min_text_size (int, optional) – Minimum width and height of the region’s axis-aligned bounding box in source-image pixels, checked before padding, clipping or warping. Regions below this threshold are skipped. Default is 5.
batch_size (int, optional) – Default inference batch size used when predict(..., batch_size=...) is not provided. Default is 128.
recognition_levels (sequence of str, optional) – Word, line or block regions to recognize. Overrides the levels in the model config; passed through **kwargs.
use_orientation (bool, optional) – Compensate known region orientation before recognition. Default is True; passed through **kwargs. Known angles take precedence over the aspect-ratio heuristic. Set rotate_threshold=None separately to disable that heuristic for regions with unknown orientation.
**kwargs – Additional options forwarded to the shared recognizer workflow.
force_download (bool)

Raises:

FileNotFoundError – If specified files do not exist.
ValueError – If weights format is invalid.

Examples

Create recognizer with default preset (auto-downloads):

>>> from manuscript.recognizers import TRBA
>>> recognizer = TRBA()

Load from local ONNX file:

>>> recognizer = TRBA(
...     weights="path/to/model.onnx",
...     config="path/to/config.json", charset="path/to/charset.txt",
... )

Load from GitHub release:

>>> recognizer = TRBA(
...     weights="github://owner/repo/v1.0/model.onnx",
...     config="github://owner/repo/v1.0/config.json",
...     charset="github://owner/repo/v1.0/charset.txt",
... )

Force CPU execution:

>>> recognizer = TRBA(device="cpu")

static train(train_csvs, train_roots, val_csvs=None, val_roots=None, *, exp_dir=None, charset_path=None, encoding='utf-8', img_h=64, img_w=256, max_len=25, hidden_size=256, num_encoder_layers=3, cnn_in_channels=3, cnn_out_channels=512, cnn_backbone='seresnet31', ctc_weight=0.3, ctc_weight_decay_epochs=50, ctc_weight_min=0.0, max_grad_norm=5.0, batch_size=32, epochs=20, lr=0.001, optimizer='AdamW', scheduler='OneCycleLR', weight_decay=0.0, momentum=0.9, val_interval=1, val_size=3000, train_proportions=None, num_workers=0, seed=42, resume_from=None, save_interval=None, device='cuda', freeze_cnn='none', freeze_enc_rnn='none', freeze_attention='none', pretrain_weights='default', recognition_levels=('word',), export_simplify=False, **extra_config)[source]

Train TRBA text recognition model on custom datasets.

Parameters:

train_csvs (str, Path or sequence of paths) – Path(s) to training CSV or TSV files. The first two columns contain the image path (relative to train_roots) and ground truth text. Use filename,text as a CSV header, or omit the header. Delimiters are detected automatically; .tsv files use tabs.
train_roots (str, Path or sequence of paths) – Root directory/directories containing training images. Must have same length as train_csvs.
val_csvs (str, Path, sequence of paths, or None, optional) – Path(s) to validation CSV files with same format as train_csvs. If None, validation samples are split from each training dataset using val_size. A None entry in a supplied list uses this fallback for the corresponding training dataset. Default is None.
val_roots (str, Path, sequence of paths, or None, optional) – Root directory/directories for validation images. Must match length of val_csvs if provided. Default is None.
exp_dir (str or Path, optional) – Experiment directory where checkpoints and logs will be saved. If None, uses the first available directory named exp1, exp2, and so on. Default is None.
charset_path (str or Path, optional) – Path to character set file. If None, uses default charset from package. Default is None.
encoding (str, optional) – Text encoding for reading CSV files. Default is "utf-8".
img_h (int, optional) – Target height for input images (pixels). Default is 64.
img_w (int, optional) – Target width for input images (pixels). Default is 256.
max_len (int, optional) – Maximum sequence length for text recognition. Default is 25.
hidden_size (int, optional) – Hidden dimension size for RNN encoder/decoder. Default is 256.
num_encoder_layers (int, optional) – Number of Bidirectional LSTM layers in the encoder. Default is 3.
cnn_in_channels (int, optional) – Number of input channels for CNN backbone. The standard training transforms and inference workflow use three-channel RGB images; keep this at 3 when using those workflows. Default is 3.
cnn_out_channels (int, optional) – Number of output channels from CNN backbone. Default is 512.
cnn_backbone ({"seresnet31", "seresnet31-lite", "seresnetlite31v2"}, optional) – CNN backbone variant. "seresnet31" keeps the standard SE-ResNet-31, while "seresnet31-lite" enables a depthwise-lite version and "seresnetlite31v2" selects SEResNet31LiteV2. Default is "seresnet31".
ctc_weight (float, optional) – Initial weight for CTC loss during training (CTC always used for stability): loss = attn_loss * (1 - ctc_weight) + ctc_loss * ctc_weight. CTC weight decays over epochs. Default is 0.3.
ctc_weight_decay_epochs (int, optional) – Number of epochs for CTC weight to decay to minimum. Default is 50.
ctc_weight_min (float, optional) – Minimum value for CTC weight after decay. Default is 0.0.
max_grad_norm (float, optional) – Maximum gradient norm for clipping (prevents gradient explosion/NaN). Default is 5.0.
batch_size (int, optional) – Training batch size. Default is 32.
epochs (int, optional) – Number of training epochs. Default is 20.
lr (float, optional) – Learning rate. Default is 1e-3.
optimizer ({"Adam", "SGD", "AdamW"}, optional) – Optimizer type. Default is "AdamW".
scheduler ({"ReduceLROnPlateau", "CosineAnnealingLR", "OneCycleLR", "None"}, optional) –
Learning rate scheduler type:
- "OneCycleLR" - one-cycle policy with cosine annealing (default, recommended)
- "ReduceLROnPlateau" - reduce LR on validation loss plateau
- "CosineAnnealingLR" - cosine annealing over epochs
- "None" or None - constant learning rate
Default is "OneCycleLR".
weight_decay (float, optional) – L2 weight decay coefficient. Default is 0.0.
momentum (float, optional) – Momentum for SGD optimizer. Default is 0.9.
val_interval (int, optional) – Perform validation every N epochs. Default is 1.
val_size (int, optional) – Number of samples held out from each training dataset when no separate validation set is supplied for it. Each such dataset must have more than val_size valid samples to leave training data. Separate validation sets are used in full. Default is 3000.
train_proportions (sequence of float, optional) – Sampling proportions for multiple training datasets, matching the length of train_csvs. Values are normalized to sum to 1.0. If None, datasets are concatenated and shuffled, so their contributions follow their sizes. Default is None.
num_workers (int, optional) – Number of data loading workers. Default is 0.
seed (int, optional) – Random seed for reproducibility. Default is 42.
resume_from (str or Path, optional) – Path to checkpoint file to resume training from. Default is None.
save_interval (int, optional) – Save the latest checkpoint every N epochs and on the final epoch. None follows validation. Best checkpoints always follow validation.
device ({"cuda", "cpu"}, optional) – Training device. Default is "cuda".
freeze_cnn ({"none", "partial", "full"}, optional) – CNN freezing policy. "partial" freezes conv0 and layers 1-3; layer4 and conv_out remain trainable. "full" freezes the entire CNN; "all" is an alias. Default is "none".
freeze_enc_rnn ({"none", "partial", "full"}, optional) – Encoder RNN freezing policy. "partial" freezes the first BiLSTM block. "full" freezes the entire encoder RNN; "all" is an alias. Default is "none".
freeze_attention ({"none", "partial", "full"}, optional) – Attention module freezing policy. "partial" freezes attention_cell for the attention decoder; for PARSeq it freezes all decoder parameters except generator. "full" freezes the entire module; "all" is an alias. Default is "none".
pretrain_weights (str, Path, bool, or None, optional) –
Pretrained weights to initialize from:
- "default" or True - use the registry checkpoint for trba_lite_g1
- None or False - train from scratch
- str/Path - path or URL to custom weights file
Default is "default".
recognition_levels (sequence of str, optional) – Word, line or block regions to recognize during inference. Saved in the model config; does not change the training samples or create region geometry. Default is ("word",).
export_simplify (bool, optional) – Simplify the automatically exported ONNX graph with onnxsim. Default is False: native simplifier crashes in some environments cannot be caught as Python exceptions. Model validation remains enabled.
**extra_config (dict, optional) – Additional configuration parameters passed to training config.

Returns:

Best validation metrics (val_acc and val_loss) and exp_dir. Saved weights and resumable checkpoints are written to exp_dir.

Return type:

dict

Examples

Train on single dataset with validation:

>>> from manuscript.recognizers import TRBA
>>>
>>> training = TRBA.train(
...     train_csvs="data/train.csv",
...     train_roots="data/train_images",
...     val_csvs="data/val.csv",
...     val_roots="data/val_images",
...     exp_dir="./experiments/trba_exp1",
...     epochs=50,
...     batch_size=64,
...     img_h=64,
...     img_w=256,
... )
>>> print(f"Training results: {training}")

Train on multiple datasets with custom proportions:

>>> train_csvs = ["data/dataset1/train.csv", "data/dataset2/train.csv"]
>>> train_roots = ["data/dataset1/images", "data/dataset2/images"]
>>> train_proportions = [0.7, 0.3]  # 70% from dataset1, 30% from dataset2
>>>
>>> training = TRBA.train(
...     train_csvs=train_csvs,
...     train_roots=train_roots,
...     train_proportions=train_proportions,
...     val_csvs="data/val.csv",
...     val_roots="data/val_images",
...     epochs=100,
...     lr=5e-4,
...     optimizer="AdamW",
...     weight_decay=1e-4,
... )

Resume training from checkpoint:

>>> training = TRBA.train(
...     train_csvs="data/train.csv",
...     train_roots="data/train_images",
...     resume_from="experiments/trba_exp1/last_ckpt.pth",
...     epochs=100,
... )

Fine-tune from pretrained weights with frozen CNN:

>>> training = TRBA.train(
...     train_csvs="data/finetune.csv",
...     train_roots="data/finetune_images",
...     pretrain_weights="default",
...     freeze_cnn="full",
...     epochs=20,
...     lr=1e-4,
... )

Train with CTC for stability (always enabled):

>>> training = TRBA.train(
...     train_csvs="data/train.csv",
...     train_roots="data/train_images",
...     optimizer="AdamW",
...     scheduler="OneCycleLR",
...     lr=1e-3,
...     ctc_weight=0.3,
...     ctc_weight_decay_epochs=50,
...     max_grad_norm=5.0,
...     epochs=100,
... )

static export(weights_path, config_path, charset_path, output_path, opset_version=14, simplify=True)[source]

Export TRBA PyTorch model to ONNX format.

This method converts a trained TRBA model from PyTorch to ONNX format, which can be used for faster inference with ONNX Runtime. The exported model must be loaded with its matching configuration and charset: TRBA(weights="model.onnx", config="config.json", charset="charset.txt").

Parameters:

weights_path (str or Path) – Path to the PyTorch model weights file (.pth).
config_path (str or Path) – Path to the model configuration JSON file. Used to determine model architecture (img_h, img_w, max_len, hidden_size, etc.).
charset_path (str or Path) – Path to the charset file (charset.txt). Used to determine num_classes for the model.
output_path (str or Path) – Path where the ONNX model will be saved (.onnx).
opset_version (int, optional) – ONNX opset version to use for export. Default is 14.
simplify (bool, optional) – If True, applies ONNX graph simplification using onnx-simplifier to optimize the model. Requires onnx-simplifier package. Default is True.

Returns:

The ONNX model is saved to output_path.

Return type:

None

Raises:

ImportError – If required packages (torch, onnx) are not installed.
FileNotFoundError – If weights_path or config_path do not exist.

Examples

Export TRBA model to ONNX:

>>> from manuscript.recognizers import TRBA
>>> TRBA.export(
...     weights_path="experiments/best_model/best_acc_weights.pth",
...     config_path="experiments/best_model/config.json",
...     charset_path="configs/charset.txt",
...     output_path="trba_model.onnx"
... )
Loading TRBA model...
=== TRBA ONNX Export ===
Max decoding length: 40
Input size: 64x256
[OK] ONNX model saved to: trba_model.onnx

Export with custom opset:

>>> TRBA.export(
...     weights_path="model.pth",
...     config_path="config.json",
...     charset_path="charset.txt",
...     output_path="model.onnx",
...     opset_version=16,
...     simplify=False
... )

Use the exported model for inference:

>>> from manuscript.detectors import EAST
>>> from manuscript.utils import read_image
>>> recognizer = TRBA(
...     weights="trba_model.onnx", config="config.json", charset="charset.txt",
... )
>>> detector = EAST()
>>> image = read_image("page.jpg")
>>> page = detector.predict(image)
>>> result = recognizer.predict(page, image=image)

PPOCRRec

class manuscript.recognizers.PPOCRRec(weights=None, config=None, charset=None, device=None, force_download=False, rotate_threshold=1.5, region_preparer='bbox', region_preparer_options=None, min_text_size=5, batch_size=16, recognition_levels=None, use_space_char=None, rec_image_shape=None, sort_by_width=False, max_batch_pixels=None, **kwargs)[source]

Bases: BaseRecognizer

Recognize Page text regions with configurable CTC ONNX models.

With region_preparer="bbox", region_preparer_options accepts padding_x, padding_y, inset_x and inset_y as fractions of the original bbox size per side. For example, {"inset_y": 0.1} removes 10% from the top and bottom before orientation compensation. Insets must be below 0.5. Padding and inset on the same axis, or relative margins and nonzero pixel pad, cannot be combined. Page geometry is preserved; other built-in crop presets reject relative margins.

sort_by_width=True batches crops of similar widths and restores the original result order. It is opt-in because changing right padding can affect CTC predictions. max_batch_pixels optionally limits padded input pixels (batch * height * width, excluding channels), without resizing crops. A single input, including required fixed ONNX batch padding, must fit the budget; otherwise prediction raises ValueError. This is not a limit on total runtime memory. batch_size remains the maximum dynamic batch size.

Parameters:

weights (str | None)
config (str | None)
charset (str | None)
device (str | None)
force_download (bool)
rotate_threshold (float | None)
region_preparer (str | Callable[[...], Sequence[Any]])
region_preparer_options (Dict[str, Any] | None)
min_text_size (int)
batch_size (int)
recognition_levels (Sequence[str] | None)
use_space_char (bool | None)
rec_image_shape (Sequence[int] | None)
sort_by_width (bool)
max_batch_pixels (int | None)
kwargs (Any)

static create_config(source, *, recognition_levels=None, output_path=None)[source]: Create an inference JSON config; new bundles recognize lines by default.

PPOCRRec.predict(page, image=None, batch_size=None, debug_save_dir=None, profile=False, return_raw=False)

Return a Page copy with recognized text and confidence for its regions.

Without an image, return an unchanged copy. Debug crops are saved after region preparation and rotation, before model preprocessing. With return_raw=True, return a dictionary containing the Page and per-region predictions with model-specific metadata.

Return type:

Union[Page, Dict[str, Any]]

Parameters:

page (Page)
image (numpy.ndarray | str | Path | Image | None)
batch_size (int | None)
debug_save_dir (str | Path | None)
profile (bool)
return_raw (bool)

TrOCR

class manuscript.recognizers.TrOCR(weights=None, config=None, decoder=None, tokenizer=None, device=None, force_download=False, rotate_threshold=None, region_preparer='bbox', region_preparer_options=None, min_text_size=5, batch_size=1, recognition_levels=None, generation=None, decoder_with_past=None, cuda_decoding='auto', **kwargs)[source]

Bases: BaseRecognizer

Recognize text regions using an ONNX encoder/decoder bundle.

Parameters:

weights (str or Path) – Encoder ONNX file, URL or registry model key. No default preset.
config (str or Path, optional) – Bundle configuration with preprocessing, generation and companion-file paths. Required for local weights; otherwise taken from the registry.
decoder (str or Path, optional) – Decoder ONNX file or URL. Defaults to the bundle’s decoder artifact.
tokenizer (str or Path, optional) – Tokenizer JSON file or URL. Defaults to the bundle’s tokenizer artifact. Requires the manuscript-ocr[trocr] extra.
device (str, optional) – Runtime device, for example "cpu" or "cuda". Selected automatically when omitted.
force_download (bool, default False) – Download remote artifacts again even when cached.
rotate_threshold (float, optional) – Rotate crops whose height exceeds width times this threshold. None disables this shape heuristic; known orientation is still used.
region_preparer (str or callable, default "bbox") – Crop strategy: "bbox", "polygon_mask", "quad_warp" or a custom callable.
region_preparer_options (dict, optional) – Additional options for the crop preparer.
min_text_size (int, default 5) – Minimum crop side length in pixels; smaller crops are skipped.
batch_size (int, default 1) – Requested crop batch size, subject to the ONNX graphs’ batch dimensions.
recognition_levels (sequence of str, optional) – Region types to recognize: "word", "line" or "block". Defaults to recognition_levels from the bundle configuration.
generation (dict, optional) – Override generation settings: num_beams, max_length, length_penalty, no_repeat_ngram_size and early_stopping.
decoder_with_past (str or Path, optional) – Cached decoder ONNX file or URL for a kv_cache bundle. Defaults to the companion artifact declared by the bundle.
cuda_decoding ({"auto", "off", "required"}, default "auto") – Keep encoder states, logits and KV caches on CUDA using optional Torch buffers and ONNX I/O binding. Requires dynamic-batch KV-cache graphs, CUDA-enabled Torch and ORT DLPack support. Install manuscript-ocr[gpu,trocr-cuda]. Auto falls back with a warning on CUDA when unavailable; required raises instead. CPU, CoreML and Web use NumPy. CUDA floating-point kernels may produce small differences in scores.
**kwargs – Additional base recognizer options, such as use_orientation and debug_save_dir.

Examples

>>> from manuscript.recognizers import TrOCR
>>> from manuscript.utils import create_page_from_image
>>> recognizer = TrOCR(
...     weights="encoder.onnx", config="config.json", recognition_levels=["line"],
... )
>>> page, image = create_page_from_image(
...     "line.png", region_type="line", return_image=True,
... )
>>> result = recognizer.predict(page, image=image)
>>> print(result.get_text())

static create_config(source, *, recognition_levels=None, output_path=None)[source]

Set recognition levels in an existing bundle configuration.

Parameters:

source (dict or str or Path) – Bundle configuration dictionary or path to its JSON file.
recognition_levels (sequence of str, optional) – Region types: "word", "line" or "block". Uses the source value when omitted, or ["line"] if the source has no value.
output_path (str or Path, optional) – Save the resulting configuration to this JSON file.

Returns:

Updated configuration copy; the source dictionary is unchanged.

Return type:

dict

TrOCR.predict(page, image=None, batch_size=None, debug_save_dir=None, profile=False, return_raw=False)

Return a Page copy with recognized text and confidence for its regions.

Without an image, return an unchanged copy. Debug crops are saved after region preparation and rotation, before model preprocessing. With return_raw=True, return a dictionary containing the Page and per-region predictions with model-specific metadata.

Return type:

Union[Page, Dict[str, Any]]

Parameters:

page (Page)
image (numpy.ndarray | str | Path | Image | None)
batch_size (int | None)
debug_save_dir (str | Path | None)
profile (bool)
return_raw (bool)

MajorityVotingRecognizer

class manuscript.recognizers.MajorityVotingRecognizer(models, *, parallel=False, memory_policy='keep')[source]

Bases: ModelComposition

Combine independent recognizers by whole-region plurality voting.

Parameters:

models (iterable of recognizers) – Nonempty ordered collection implementing predict(page, image=None). Each model receives an independent Page copy. Only TextRegion text and recognition_confidence may change; tree structure, child order, geometry and other fields must remain unchanged.
parallel (bool, optional) – Run models concurrently in threads if True; default is False. Completion order does not change tie priority. Participants share the image array and must not modify it.
memory_policy ({"keep", "sequential"}, optional) – Keep sessions loaded by default. “sequential” releases each member’s sessions after its call, including on failure. Nested compositions inherit this policy. Requires close() on every participant and disallows parallel=True. Shared participants need external locking.

Examples

>>> from manuscript import Pipeline
>>> recognizer = MajorityVotingRecognizer([model_a, model_b, model_c])
>>> pipeline = Pipeline(recognizer=recognizer)
>>> result = pipeline.predict("document.jpg")

MedoidRecognizer

class manuscript.recognizers.MedoidRecognizer(models, *, parallel=False, memory_policy='keep')[source]

Bases: MajorityVotingRecognizer

Select the existing text hypothesis closest to all model responses.

Parameters:

models (iterable of recognizers) – Nonempty ordered collection implementing predict(page, image=None). Each model receives an independent Page copy. Only TextRegion text and recognition_confidence may change; other changes raise an error.
parallel (bool, optional) – Run models concurrently in threads if True; default is False. List order determines tie priority regardless of completion order.
memory_policy ({"keep", "sequential"}, optional) – Keep sessions loaded by default. “sequential” releases participant sessions after each call, including on failure, and applies to nested compositions. Requires close() and parallel=False; shared participants need external synchronization.

Examples

>>> recognizer = MedoidRecognizer([model_a, model_b, model_c])
>>> result = recognizer.predict(page, image=image, return_raw=True)
>>> scores = result["voting"][0]["distance_sums"]