Detectors

Supported models:

ConsensusWBFDetector

class manuscript.detectors.ConsensusWBFDetector(detectors, *, weights=None, cluster_iou=0.5, consensus_fraction=0.5, minimum_votes=2, memory_policy='keep')[source]

Bases: ModelComposition

Fuse detections supported by enough independent participants.

detectors is a nonempty ordered iterable of objects implementing predict(image) -> Page (or a dictionary containing page). Their Pages must use original-image coordinates and dimensions. All geometric descendants are treated as flat detections, grouped by (type, label). Configure equivalent labels on participants to let their objects match.

weights=None uses equal arithmetic means, reproducing consensus box fusion. Otherwise supply one finite positive weight per detector. Weights affect cluster representatives and fused coordinates, never vote counts. Detection confidence is not used. Output confidence remains None.

Detectors run in supplied order. At each step, cluster representatives are frozen and IoU pairs are greedily matched one-to-one. Ties sort by descending (IoU, box index, cluster index). Each detector supplies at most one vote per cluster. Empty answers do not reduce the required vote count; errors abort. No additional NMS is applied. The algorithm depends on participant order.

cluster_iou is in [0, 1], consensus_fraction in [0, 1], and minimum_votes is a positive integer. Required votes are the maximum of minimum_votes and ceil(number of detectors * consensus_fraction). A threshold above the number of detectors is allowed and yields an empty Page.

memory_policy follows Pipeline/recognizer compositions: “keep” (default) or “sequential”; the latter requires close() on every participant. Pickle support preserves nested participants, order, weights and parameters, provided participants themselves support pickling.

property required_votes
predict(image, return_raw=False, *, return_masks=False, return_outputs=False)[source]

Return a flat Page, or standard raw detection data with vote metadata.

Any return flag enables dictionary output. Fusion produces no masks or model tensors. Raw detections include box, polygon, type, label, votes, and detector_indices; indices refer to the supplied participant order.

predict_batch(images, **kwargs)[source]

Fuse each image independently, in input order; an empty iterable returns [].

YOLO

class manuscript.detectors.YOLO(weights=None, config=None, device=None, force_download=False, *, score_thresh=0.1, class_ids=None, target_size=None, axis_aligned_output=None, containment_threshold=0.9, **kwargs)[source]

Bases: BaseDetector

Initialize YOLO text detector with ONNX Runtime.

Parameters:

  • weights (str or Path, optional) –

    Path or identifier for ONNX model weights. Supports:

    • Local file path: "path/to/model.onnx"

    • HTTP/HTTPS URL: "https://example.com/model.onnx"

    • GitHub release: "github://owner/repo/tag/file.onnx"

    • Google Drive: "gdrive:FILE_ID"

    • Preset name: "yolo26s_obb_text_g1" or "yolo26x_obb_text_g1"

    • None: auto-downloads default preset (yolo26x_obb_text_g1)

    The ONNX model may return either standard detections in xyxy, score, class_id format with shape [N, 6] / [1, N, 6] or oriented detections in cx, cy, w, h, score, class_id, angle format with shape [N, 7] / [1, N, 7].

  • config (dict, str or Path, optional) – Dictionary, path or identifier for model configuration JSON/YAML. Same URL schemes as weights. If None, uses the config artifact from the model registry. Local weights require an explicit config. Required region_classes maps model class IDs to element fields, e.g. {0: {"type": "word"}, 1: {"type": "table"}}. Detections are returned as flat page children; layout establishes hierarchy.

  • device (str, optional) –

    Compute device: "cuda", "coreml", or "cpu". If None, selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. For GPU/CoreML acceleration:

    • CUDA (NVIDIA): pip install onnxruntime-gpu

    • CoreML (Apple Silicon): included in standard macOS ONNX Runtime wheels; select device="coreml" explicitly.

    Default is None (automatic selection).

  • score_thresh (float, optional) – Confidence threshold applied to model outputs after ONNX inference and before the additional containment cleanup pass. Default is 0.1.

  • class_ids (sequence of int or None, optional) – Optional whitelist of class IDs to keep. If None, all classes are kept. Default is None.

  • target_size (int or None, optional) – Square inference size used for letterbox preprocessing. Images are resized into (target_size, target_size) before ONNX inference. If None, reads imgsz from the explicitly supplied config or the preset’s config artifact. Neighboring files are not discovered. Configurations without imgsz fall back to 1280.

  • axis_aligned_output (bool, optional, deprecated) – Deprecated. If explicitly True, OBB detections are converted to standard axis-aligned rectangles in the Page. By default, Page regions retain rotated polygons. With return_raw=True, each entry in detections retains its native polygon and a box row in cx, cy, w, h, score, class_id, angle format. For non-OBB models this flag has no effect.

  • containment_threshold (float or None, optional) – Removes a smaller box when at least this fraction of its area is covered by a larger box of the same class. For example, 0.9 removes boxes that are contained by 90% or more. Set to None to disable this extra cleanup. Default is 0.9.

  • force_download (bool)

static create_config(source=None, *, imgsz=None, region_classes=None, output_path=None)[source]

Build an inference JSON config from training YAML/JSON or explicit values.

Only input size and class mapping are retained. Class 0 defaults to words; provide region_classes for other models. Returns a dictionary usable as YOLO(config=...) and optionally saves it to output_path.

Example:

YOLO.create_config("args.yaml", output_path="model.json")
YOLO.create_config(imgsz=1024,
                   region_classes={0: {"type": "line"}})
Return type:

Dict[str, Any]

predict(img_or_path, return_raw=False, *, return_masks=False, return_outputs=False)[source]

Run YOLO ONNX inference on a single image and return detected page structure.

Parameters:

  • img_or_path (str or pathlib.Path or numpy.ndarray) – Path to an image file or an RGB image provided as a NumPy array with shape (H, W, 3) in uint8 format.

  • return_raw (bool, optional) – Return a dictionary with the Page and decoded detections.

  • return_masks (bool, optional) – Enable dictionary output. YOLO does not generate masks.

  • return_outputs (bool, optional) – Include original named ONNX output tensors and enable dictionary output.

Returns:

Flat document tree with original image dimensions and elements whose types and labels come from region_classes. With any result flag enabled, returns page, detections, image_size and metadata, plus outputs when requested.

Return type:

Page or dict

Examples

Run inference and get structured output:

>>> from manuscript.detectors import YOLO
>>> model = YOLO(weights="yolo26x_obb_text_g1")
>>> page = model.predict("page.jpg")
>>> print(page.children[0].detection_confidence)

EAST

class manuscript.detectors.EAST(weights=None, device=None, force_download=False, *, config=None, target_size=1280, expand_ratio_w=1.4, expand_ratio_h=1.5, expand_power=0.6, score_thresh=0.6, iou_threshold=0.05, iou_threshold_standard=0.05, score_geo_scale=0.25, quantization=2, axis_aligned_output=None, remove_area_anomalies=False, anomaly_sigma_threshold=5.0, anomaly_min_box_count=30, use_tta=False, tta_iou_thresh=0.1, **kwargs)[source]

Bases: BaseDetector

Initialize EAST text detector with ONNX Runtime.

Parameters:

  • weights (str or Path, optional) –

    Path or identifier for ONNX model weights. Supports:

    • Local file path: "path/to/model.onnx"

    • HTTP/HTTPS URL: "https://example.com/model.onnx"

    • GitHub release: "github://owner/repo/tag/file.onnx"

    • Google Drive: "gdrive:FILE_ID"

    • Preset name: "east_50_g1"

    • None: auto-downloads default preset (east_50_g1)

  • device (str, optional) –

    Compute device: "cuda", "coreml", or "cpu". If None, selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. For GPU/CoreML acceleration:

    • CUDA (NVIDIA): pip install onnxruntime-gpu

    • CoreML (Apple Silicon): included in standard macOS ONNX Runtime wheels; select device="coreml" explicitly.

    Default is None (automatic selection).

  • config (dict or path, optional) – Region mapping configuration. Required key region_classes maps class IDs to element fields, e.g. {"0": {"type": "word"}}. If omitted, uses a registry config artifact. Local weights require an explicit config path.

  • target_size (int, optional) – Input image size for inference. Images are resized to (target_size, target_size). Default is 1280.

  • expand_ratio_w (float, optional) – Horizontal expansion factor applied to detected boxes after NMS. Default is 1.4.

  • expand_ratio_h (float, optional) – Vertical expansion factor applied to detected boxes after NMS. Default is 1.5.

  • expand_power (float, optional) – Power for non-linear box expansion. Controls how expansion scales with box size. - 1.0 = linear (small and large boxes expand equally) - <1.0 = small boxes expand more (e.g., 0.5, recommended for character-level detection) - >1.0 = large boxes expand more Default is 0.6.

  • score_thresh (float, optional) – Confidence threshold for selecting candidate detections before NMS. Default is 0.6.

  • iou_threshold (float, optional) – IoU threshold for locality-aware NMS merging phase. Default is 0.05.

  • iou_threshold_standard (float, optional) – IoU threshold for standard NMS after locality-aware merging. If None, uses the same value as iou_threshold. Default is 0.05.

  • score_geo_scale (float, optional) – Scale factor for decoding geometry/score maps. Default is 0.25.

  • quantization (int, optional) – Quantization resolution for point coordinates during decoding. Default is 2.

  • axis_aligned_output (bool, optional, deprecated) – Deprecated. Explicit True outputs axis-aligned rectangles. By default, original quads are preserved.

  • remove_area_anomalies (bool, optional) – If True, removes quads with extremely large area relative to the distribution. Default is False.

  • anomaly_sigma_threshold (float, optional) – Sigma threshold for anomaly area filtering. Default is 5.0.

  • anomaly_min_box_count (int, optional) – Minimum number of boxes required before anomaly filtering. Default is 30.

  • use_tta (bool, optional) – Enable Test-Time Augmentation (TTA). When enabled, inference is run on both the original and horizontally flipped image. Only original detections with a matching box in the flipped view are retained. Default is False.

  • tta_iou_thresh (float, optional) – IoU threshold for matching boxes from original and flipped images during TTA. An original detection is retained when a flipped-view box has IoU > threshold. Default is 0.1.

  • force_download (bool)

predict(img_or_path, return_raw=False, *, return_masks=False, return_outputs=False)[source]

Run EAST inference and return detected page structure.

Parameters:

  • img_or_path (str or pathlib.Path or numpy.ndarray) – Path to an image file or an RGB image provided as a NumPy array with shape (H, W, 3) in uint8 format.

  • return_raw (bool)

  • return_masks (bool)

  • return_outputs (bool)

Returns:

Detection result as a flat document tree of configured text regions. Each region has polygon coordinates and a detection confidence score. Child sequence defines order; a layout stage can establish reading order.

Return type:

Page

Examples

Perform inference and get structured output:

>>> from manuscript.detectors import EAST
>>> model = EAST()
>>> img_path = r"example/ocr_example_image.jpg"
>>> page = model.predict(img_path)
>>> # Access the first detected region, when present
>>> for region in page.children[:1]:
...     print(f"Confidence: {region.detection_confidence}")

Visualize results separately:

>>> from manuscript.utils import visualize_page, read_image
>>> page = model.predict(img_path)
>>> img = read_image(img_path)
>>> vis_img = visualize_page(img, page)
>>> vis_img.show()
static train(train_images, train_anns, val_images, val_anns, *, experiment_root='./experiments', model_name='resnet_quad', backbone_name='resnet50', pretrained_backbone=True, freeze_first=False, target_size=1024, score_geo_scale=None, score_map_shrink_ratio=0.3, epochs=500, batch_size=3, accumulation_steps=1, lr=0.0001, lr_scheduler='cosine_restart', lr_scheduler_params=None, augmentation_config=None, grad_clip=5.0, early_stop=100, use_sam=False, sam_type='asam', use_lookahead=False, use_ema=False, use_multiscale=True, use_ohem=False, ohem_ratio=0.5, use_focal_geo=True, focal_gamma=2.0, resume_from=None, val_interval=1, num_workers=0, log_collage=True, detection_level='word', device=None)[source]

Train EAST model on custom datasets.

Parameters:

  • train_images (str, Path or sequence of paths) – Path(s) to training image folders.

  • train_anns (str, Path or sequence of paths) – Path(s) to COCO-format JSON annotation files corresponding to train_images.

  • val_images (str, Path or sequence of paths) – Path(s) to validation image folders.

  • val_anns (str, Path or sequence of paths) – Path(s) to COCO-format JSON annotation files corresponding to val_images.

  • experiment_root (str, optional) – Base directory where experiment folders will be created. Default is "./experiments".

  • model_name (str, optional) – Folder name inside experiment_root for logs and checkpoints. Default is "resnet_quad".

  • backbone_name ({"resnet50", "resnet101"}, optional) –

    Backbone architecture to use. Options:

    • "resnet50" — ResNet-50 (faster, less parameters)

    • "resnet101" — ResNet-101 (slower, more capacity)

    Default is "resnet50".

  • pretrained_backbone (bool, optional) – Use ImageNet-pretrained backbone weights. Default True.

  • freeze_first (bool, optional) – Freeze lowest layers of the backbone. Default False.

  • target_size (int, optional) – Resize images to (target_size, target_size) without preserving aspect ratio. Default 1024.

  • score_geo_scale (float, optional) – Multiplier to recover original coordinates from score/geo maps. If None, automatically taken from the model. Default None.

  • score_map_shrink_ratio (float, optional) – Shrink ratio for score map. Default 0.3.

  • epochs (int, optional) – Number of training epochs. Default 500.

  • batch_size (int, optional) – Batch size per GPU. Default 3.

  • accumulation_steps (int, optional) –

    Number of gradient accumulation steps. Effective batch size will be batch_size * accumulation_steps. Use this to train with larger effective batch sizes when GPU memory is limited. For example:

    • batch_size=2, accumulation_steps=4 → effective batch size = 8

    • batch_size=1, accumulation_steps=8 → effective batch size = 8

    Default is 1 (no accumulation).

  • lr (float, optional) – Learning rate. Default 1e-4.

  • lr_scheduler ({"cosine_restart", "cosine", "linear", "step", "exponential", "plateau", "none"}, optional) – Learning rate scheduler type. Default "cosine_restart".

  • lr_scheduler_params (dict, optional) –

    Extra scheduler parameters (depends on scheduler type). Common keys:

    • cosine_restart: t0, t_mult, eta_min

    • cosine: t_max, eta_min

    • linear: final_factor

    • step: step_size, gamma

    • exponential: gamma

    • plateau: factor, patience, min_lr

  • augmentation_config (dict, optional) –

    Full augmentation configuration passed to EASTDataset. This is the preferred place for all augmentation settings. Any keys here override the corresponding defaults. Supported keys include:

    • quad_source for COCO polygon-to-quad conversion: "auto" preserves 4-point polygons and falls back to "min_area_rect" for longer polygons, "as_is" accepts only 4-point polygons, and "min_area_rect" always fits the minimum-area rectangle

    • flip_prob, color_jitter

    • small_rotate_prob, small_rotate_deg

    • perspective_prob, perspective_scale

    • blur_prob, blur_ksize_range

    • noise_prob, noise_std, salt_pepper_prob

    • jpeg_prob, jpeg_quality_range

    • shading_prob, shading_strength

    • gamma_prob, gamma_range

    • downscale_prob, downscale_range

    • negative_prob

  • grad_clip (float, optional) – Gradient clipping value (L2 norm). Default 5.0.

  • early_stop (int, optional) – Patience (epochs without improvement) for early stopping. Default 100.

  • use_sam (bool, optional) – Enable SAM optimizer. Default False.

  • sam_type ({"sam", "asam"}, optional) – Variant of SAM to use. Default "asam".

  • use_lookahead (bool, optional) – Wrap optimizer with Lookahead. Default False.

  • use_ema (bool, optional) – Maintain EMA version of model weights. Default False.

  • use_multiscale (bool, optional) – Random multi-scale training. Default True.

  • use_ohem (bool, optional) – Online Hard Example Mining. Default False.

  • ohem_ratio (float, optional) – Ratio of hard negatives for OHEM. Default 0.5.

  • use_focal_geo (bool, optional) – Apply focal loss to geometry channels. Default True.

  • focal_gamma (float, optional) – Gamma for focal geometry loss. Default 2.0.

  • resume_from (str or Path, optional) – Resume training from a previous experiment or initialize from local weights: a) experiment directory, b) …/checkpoints/, c) direct path to full-state checkpoint (last_state.pt). If a weights-only file is provided (e.g. best.pth), model weights are loaded, but training starts in the current experiment_root/model_name directory. Default None.

  • val_interval (int, optional) – Run validation every N epochs. Default 1.

  • num_workers (int, optional) – Number of worker processes for data loading. Set to 0 for single-process loading (safer on Windows). Default 0.

  • log_collage (bool, optional) – Whether to generate and log validation collage images to TensorBoard. Disable to save memory on GPUs with limited VRAM. Default True.

  • device (torch.device, optional) – CUDA or CPU device. Auto-selects if None.

  • detection_level (str)

Returns:

Model after the last completed training epoch (EMA if enabled, otherwise base model). Best validation weights are saved separately as best_loss.pth and best_dice.pth in the experiment’s checkpoints directory.

Return type:

torch.nn.Module

Examples

Train on two datasets with validation:

>>> from manuscript.detectors import EAST
>>>
>>> train_images = [
...     "/data/archive/train_images",
...     "/data/ddi/train_images"
... ]
>>> train_anns = [
...     "/data/archive/train.json",
...     "/data/ddi/train.json"
... ]
>>> val_images = [
...     "/data/archive/test_images",
...     "/data/ddi/test_images"
... ]
>>> val_anns = [
...     "/data/archive/test.json",
...     "/data/ddi/test.json"
... ]
>>>
>>> trained_model = EAST.train(
...     train_images=train_images,
...     train_anns=train_anns,
...     val_images=val_images,
...     val_anns=val_anns,
...     backbone_name="resnet50",
...     target_size=256,
...     epochs=20,
...     batch_size=4,
...     use_sam=False,
...     freeze_first=False,
...     val_interval=3,
... )
>>> print("Model after the last completed epoch:", trained_model)

Configure augmentations via augmentation_config:

>>> aug_cfg = {
...     "quad_source": "auto",
...     "flip_prob": 0.02,
...     "color_jitter": (0.1, 0.1, 0.1, 0.05),
...     "small_rotate_prob": 0.15,
...     "small_rotate_deg": 1.5,
...     "perspective_prob": 0.08,
...     "perspective_scale": 0.015,
...     "blur_prob": 0.1,
...     "blur_ksize_range": (3, 5),
...     "noise_prob": 0.1,
...     "noise_std": 0.008,
...     "salt_pepper_prob": 0.0005,
...     "jpeg_prob": 0.1,
...     "jpeg_quality_range": (75, 95),
...     "shading_prob": 0.1,
...     "shading_strength": 0.1,
...     "gamma_prob": 0.2,
...     "gamma_range": (0.95, 1.05),
...     "downscale_prob": 0.1,
...     "downscale_range": (0.7, 0.95),
...     "negative_prob": 0.05,
... }
>>> trained_model = EAST.train(
...     train_images=train_images,
...     train_anns=train_anns,
...     val_images=val_images,
...     val_anns=val_anns,
...     augmentation_config=aug_cfg,
... )
static export(weights_path, output_path, backbone_name=None, input_size=1280, opset_version=14, simplify=True, detection_level='word')[source]

Export EAST PyTorch model to ONNX format.

This method converts a trained EAST model from PyTorch to ONNX format, which can be used for faster inference with ONNX Runtime. The exported model can be loaded using EAST(weights="model.onnx", config="model.json"). The exporter writes the matching configuration alongside the ONNX graph.

Parameters:

  • weights_path (str or Path) – Path to the PyTorch model weights file (.pth).

  • output_path (str or Path) – Path where the ONNX model will be saved (.onnx).

  • backbone_name ({"resnet50", "resnet101"}, optional) – Backbone architecture of the model. If None, will be automatically detected from the checkpoint. Must match the architecture used during training. Default is None (auto-detect).

  • input_size (int, optional) – Input image size (height and width). The model will accept images of shape (batch, 3, input_size, input_size). Default is 1280.

  • opset_version (int, optional) – ONNX opset version to use for export. Default is 14.

  • simplify (bool, optional) – If True, applies ONNX graph simplification using onnx-simplifier to optimize the model. Requires onnx-simplifier package. Default is True.

  • detection_level (str)

Returns:

The ONNX model is saved to output_path.

Return type:

None

Raises:

  • ImportError – If required packages (torch, onnx) are not installed.

  • FileNotFoundError – If weights_path does not exist.

  • ValueError – If backbone_name doesn’t match the checkpoint architecture.

Notes

The exported ONNX model has two outputs:

  • score_map: Text confidence map with shape (batch, 1, H, W)

  • geo_map: Geometry map with shape (batch, 8, H, W)

The model supports dynamic batch size and image dimensions through dynamic axes configuration.

Automatic Backbone Detection:

The method checks the checkpoint’s backbone.extractor.layer3 keys for layer3.10. Its presence selects ResNet-101; otherwise, the exporter selects ResNet-50.

Examples

Export with automatic backbone detection:

>>> from manuscript.detectors import EAST
>>> EAST.export(
...     weights_path="east_resnet50.pth",
...     output_path="east_model.onnx"
... )
Auto-detected backbone: resnet50
Exporting to ONNX (opset 14)...
[OK] ONNX model saved to: east_model.onnx

Export with explicit backbone:

>>> EAST.export(
...     weights_path="custom_weights.pth",
...     output_path="custom_model.onnx",
...     backbone_name="resnet101",
...     input_size=1024,
...     simplify=False
... )

Use the exported model for inference:

>>> detector = EAST(
...     weights="east_model.onnx",
...     config="east_model.json",
...     device="cuda"
... )
>>> result = detector.predict("image.jpg")

See also

EAST.__init__

Initialize EAST with ONNX weights and matching configuration.

PPOCR

class manuscript.detectors._ppocr.PPOCR(weights, config=None, device=None, threshold=None, box_threshold=None, unclip_ratio=None, limit_side_len=None, limit_type=None, force_download=False, *, runtime='auto', web_external_data=None)[source]

Bases: BaseDetector

Initialize a PP-OCR DB text detector with ONNX Runtime.

Parameters:

  • weights (str or Path) – Compatible ONNX model file or model registry key. Local files, HTTP/HTTPS URLs, github://owner/repo/tag/file.onnx and gdrive:FILE_ID are supported. There is no default preset.

  • config (dict, str or Path, optional) – DB inference configuration, or a path/identifier for its JSON/YAML file. Required for local weights; registry presets supply their config. Must contain schema_version=1, task="text_detection", algorithm="DB", and matching preprocess/postprocess settings. region_classes must map foreground class 0 to document fields, e.g. {0: {"type": "line"}}. Use create_config() to prepare a config from a supported Paddle inference YAML or exported JSON.

  • device ({"cuda", "coreml", "cpu"}, optional) – Execution device. None selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. Select CoreML explicitly on supported macOS installations. Default is None.

  • threshold (float or None, optional) – Pixel probability threshold in [0, 1] for forming foreground contours. None uses postprocess.threshold from the config.

  • box_threshold (float or None, optional) – Minimum mean probability within a candidate polygon, in [0, 1]. None uses postprocess.box_threshold from the config.

  • unclip_ratio (float or None, optional) – Nonnegative polygon expansion ratio. Larger values expand detected regions further. None uses postprocess.unclip_ratio.

  • limit_side_len (int or None, optional) – Resize reference length in pixels; must be at least 32. Its meaning depends on limit_type. None uses preprocess.limit_side_len.

  • limit_type ({"min", "max", "resize_long"} or None, optional) – Resize rule: "min" enlarges a short side below the reference; "max" shrinks a long side above it; "resize_long" scales the long side to the reference. None uses preprocess.limit_type.

  • force_download (bool, optional) – Download artifacts again instead of reusing cached files. Default is False.

Examples

Load compatible local weights and adjust detection thresholds:

>>> from manuscript.detectors._ppocr import PPOCR
>>> detector = PPOCR(
...     weights="model.onnx", config="config.json",
...     threshold=0.3, box_threshold=0.6, device="cpu",
... )
>>> page = detector.predict("document.jpg")
predict(img_or_path, return_raw=False, *, return_masks=False, return_outputs=False)[source]

Return flat configured regions with full DB polygons in original image coordinates.

static create_config(source, *, region_classes=None, output_path=None)[source]

Create inference JSON from an existing config or a local HF model folder.

DB exposes one foreground class (0), defaulting to line. Supply region_classes={0: {"type": "word"}} for word-oriented models. Training dependencies are not needed to create the configuration.

RFDETR

class manuscript.detectors._rfdetr.RFDETR(weights, config=None, device=None, score_thresh=None, class_ids=None, target_size=None, force_download=False, mask_threshold=None, *, runtime='auto', web_external_data=None)[source]

Bases: BaseDetector

Initialize RF-DETR detection or instance segmentation with ONNX Runtime.

Parameters:

  • weights (str or Path) – Compatible ONNX model file or model registry key. Local files, HTTP/HTTPS URLs, github://owner/repo/tag/file.onnx and gdrive:FILE_ID are supported. There is no default preset.

  • config (dict, str or Path, optional) – Inference configuration, or a path/identifier for its JSON/YAML file. Required for local weights; registry presets supply their config. class_names maps output class IDs to names, and region_classes maps selected IDs to document fields, e.g. {0: {"type": "word"}}. Preprocessing and tensor names must match the exported model.

  • device ({"cuda", "coreml", "cpu"}, optional) – Execution device. None selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. Select CoreML explicitly on supported macOS installations. Default is None.

  • score_thresh (float or None, optional) – Minimum detection confidence in [0, 1]. None uses the config’s score_thresh, falling back to 0.5 when absent.

  • class_ids (sequence of int or None, optional) – Class IDs to retain. Each must appear in both class_names and region_classes. None uses the config’s class_ids, or all mapped region IDs when that setting is absent.

  • target_size (int or None, optional) – Square input size in pixels, without aspect-ratio preservation. None uses the config’s target_size or the fixed ONNX input dimensions. Dynamic image dimensions require an explicit size. A supplied size must agree with a fixed ONNX input shape.

  • force_download (bool, optional) – Download artifacts again instead of reusing cached files. Default is False.

  • mask_threshold (float or None, optional) – Foreground probability threshold for instance masks, strictly between 0 and 1. None uses the config’s mask_threshold, falling back to 0.5 when absent. Applies only to models with mask outputs.

Examples

Load compatible local weights with their matching inference config:

>>> from manuscript.detectors._rfdetr import RFDETR
>>> detector = RFDETR(
...     weights="model.onnx", config="config.json",
...     score_thresh=0.4, device="cpu",
... )
>>> page = detector.predict("document.jpg")
predict(image, return_raw=False, *, return_masks=False, return_outputs=False)[source]

Return text Page or compact detections; masks/tensors are opt-in.

static create_config(source, *, region_classes=None, output_path=None)[source]

Preserve inference settings and add structural classes and angles.

Class semantics are supplied explicitly, not inferred from class names. The deprecated text-only filter is removed; all mapped classes are included unless the source has an explicit class_ids filter.

Mask2Former

class manuscript.detectors._mask2former.Mask2Former(weights=None, config=None, device=None, force_download=False, *, score_threshold=None, mask_logit_threshold=None, nms_iou_threshold=None, min_mask_area=None, **kwargs)[source]

Bases: BaseDetector

Initialize Mask2Former instance segmentation with ONNX Runtime.

Parameters:

  • weights (str or Path, optional) – Compatible ONNX model file or model registry key. Local files, HTTP/HTTPS URLs, github://owner/repo/tag/file.onnx and gdrive:FILE_ID are supported. None selects the default preset mask2former_line_v0_prev and downloads its supporting artifacts.

  • config (dict, str or Path, optional) – Inference configuration, or a path/identifier for its JSON/YAML file. Required for local weights; registry presets supply their config. region_classes maps output class IDs to document fields, e.g. {0: {"type": "line"}}. Image normalization, input size and tensor names must match the exported graph. create_config() builds a config dictionary with standard defaults and an explicit class mapping.

  • device ({"cuda", "coreml", "cpu"}, optional) – Execution device. None selects CUDA when ONNX Runtime reports GPU support, otherwise CPU. Select CoreML explicitly on supported macOS installations. Default is None.

  • force_download (bool, optional) – Download artifacts again instead of reusing cached files. Default is False.

  • score_threshold (float or None, optional) – Minimum confidence in [0, 1], applied to the class probability and the final score (class probability times mean foreground-mask probability). None uses the config value, or 0.15 when absent.

  • mask_logit_threshold (float or None, optional) – Finite logit threshold for creating binary masks, before sigmoid. None uses the config value, or 0.0 when absent.

  • nms_iou_threshold (float or None, optional) – Mask IoU threshold in [0, 1] for suppressing overlapping detections of the same class. None uses the config value, or 0.5 when absent.

  • min_mask_area (int or None, optional) – Minimum foreground area in source-image pixels; must be a positive integer. None uses the config value, or 16 when absent.

  • **kwargs – Additional options passed to BaseArtifactModel.

Examples

Load the text-line preset:

>>> from manuscript.detectors._mask2former import Mask2Former
>>> detector = Mask2Former(weights="mask2former_line_v0_prev", device="cpu")
>>> page = detector.predict("document.jpg")

Load compatible local weights with their matching config:

>>> detector = Mask2Former(weights="model.onnx", config="config.json")
static create_config(*, image_size=1024, mask_interpolation_size=384, image_mean=(0.485, 0.456, 0.406), image_std=(0.229, 0.224, 0.225), score_threshold=0.15, mask_logit_threshold=0.0, nms_iou_threshold=0.5, min_mask_area=16, region_classes=None, output_path=None)[source]

Build an inference config; class 0 defaults to text lines.

Optionally save JSON to output_path. The returned dictionary can also be passed directly to Mask2Former(config=...).

predict(image, return_raw=False, *, return_masks=False, return_outputs=False)[source]