Skip to content

LWIR Tile Validator

PyQt6 desktop application for systematic visual validation of HA-to-mosaic tile alignment quality. The human-in-the-loop QC step between LWIR-Align's registration output and PIUnet's training data.

What It Does

The validator displays a mosaic tile (384x384 or 192x192 pixels) alongside the top 8-9 HA image patches warped into that tile's coordinate space using GPU shaders. A human reviewer navigates tile-by-tile, annotating each patch as Good / Problematic / Needs Review. Completed annotations drive the ProbaV dataset export -- only tiles marked "good" become training examples.

The tool handles 1,364+ tiles across 4 configurations (Mosaic A/B x 384/192 tile sizes), with 210 valid 384x384 tiles as the primary validation target. All tile metadata, HA images, and homography matrices are read from LWIR-Align's output directory.

Source: lwir_tile_validator/README.md

Architecture

MVC with 80% Hephaestus Reuse

The application follows Model-View-Controller, with the GPU rendering pipeline directly copied from the Hephaestus project:

TileMetadataLoader -> TileNavigator -> TileValidationController
                                              |
                      +-------------------+---+-------------------+
                      |                                           |
             HAPatchGrid (GPU warping)              AnnotationController
                      |                                           |
             OpenGL Shaders                         AnnotationManager -> JSON

Reused from Hephaestus (~2,000 lines, unchanged): - geometry.py -- Homography analysis and transformation utilities - viewport.py -- 2D viewport management (zoom, pan, coordinate conversion) - texture_mgr.py -- OpenGL texture upload and management - shader_renderer.py -- Modern OpenGL shader pipeline for GPU warping - renderer.py -- Basic OpenGL setup and projection utilities

Key architectural difference from Hephaestus: Hephaestus is matrix-centric (refining individual HA-to-mosaic homographies), while the tile validator is tile-centric (validating the quality of multiple HA patches within a fixed mosaic region).

Source: lwir_tile_validator/docs/HEPHAESTUS_INTEGRATION.md, lwir_tile_validator/CLAUDE.md

Module Structure

src/
  models/         # Data structures and persistence
    tile_data.py        # TileData, HAImageData, TileMetadataLoader
    annotation.py       # PatchAnnotation, TileAnnotation, ValidationSession
  services/       # Business logic
    tile_navigator.py   # Navigation, filtering, config switching
    tile_image_manager.py  # Image loading with LRU caching (50 images)
  frontend/       # UI and rendering
    main_window.py      # Main application window coordinator
    ha_patch_grid.py    # 8/9-patch grid with GPU rendering
    image_widget.py     # Individual patch display widget
    export/
      probav_exporter.py  # ProbaV-format dataset export
    keyboard/
      shortcut_handler.py  # Keyboard navigation
  config.py       # Configurable path management
  project_paths.py  # Centralized absolute paths

Source: lwir_tile_validator/CLAUDE.md

Features

Real-Time GPU Rendering

The GPU pipeline provides 60fps rendering with real-time homography warping of multiple textures simultaneously. For each tile:

  1. Load 8-9 HA images for the current tile
  2. Upload as OpenGL textures
  3. Apply homography transformations via vertex shader
  4. Render warped patches in a 2x4 or 3x3 grid layout

Performance targets: <500ms tile navigation, <2s for loading and warping 8x384x384 patches, <2GB RAM, <1GB VRAM.

Source: lwir_tile_validator/docs/REQUIREMENTS.md

TileNavigator (adapted from Hephaestus SimpleIndexBrowser) loads metadata for all 4 configurations on startup and filters tiles by minimum HA image count (default: 1). Navigation is keyboard-driven:

  • Arrow Left/Right: Previous/Next tile
  • Home/End: First/Last tile
  • Space/Enter: Advance to next tile
  • Number keys 1-3: Quick annotate (Good / Problematic / Needs Review)
  • Ctrl+S: Save session

Source: lwir_tile_validator/src/services/tile_navigator.py

Three-Level Annotation Hierarchy

Defined in src/models/annotation.py:

  1. PatchAnnotation -- Per-HA-patch: status (good / problematic / needs_review), issue tags (misalignment, distortion, occlusion), notes, timestamp
  2. TileAnnotation -- Per-tile aggregate: auto-computes status from child patches (any problematic patch makes the tile problematic), reviewer notes, excluded timestamps for swapped-out patches
  3. ValidationSession -- Session persistence: session ID, current config, current tile index, progress counts, all tile annotations

The tile status auto-derives from patch annotations: if any patch is problematic, the tile is problematic; if any needs review, the tile needs review; otherwise good.

Source: lwir_tile_validator/src/models/annotation.py

Session Management

  • Auto-saves every 30 seconds via QTimer
  • Saves on navigation events (tile change) and annotation events (patch status change)
  • Session file: tile_validation_session.json in working directory
  • Resume from last position on restart

Source: lwir_tile_validator/CLAUDE.md

Patch Swapping and Sharpness Scoring

Later addition (commit a9380e8): ability to swap out poor-quality patches and score patches by sharpness. This lets the reviewer replace a blurry or misaligned HA patch with a better alternative from the available frames for that tile.

9-Patch Grid Support

Extended from the original 8-patch (2x4) layout to also support 9-patch (3x3) grids (commits 29c4d79, 03e0f98). The system always creates 9 widget slots and shows/hides the 9th as needed.

Source: lwir_tile_validator/src/frontend/ha_patch_grid.py

ProbaV Dataset Export

The ProbaVExporter (src/frontend/export/probav_exporter.py) exports validated tiles in a format compatible with the ProbaV super-resolution benchmark structure:

probav_exports/mosaicA_384_TIMESTAMP/
  tile_103/
    HR.png          # 384x384 mosaic patch (high-resolution reference)
    SM.png          # Status mask for HR patch
    LR000.png       # 128x128 HA patch (low-resolution view)
    LR001.png       # ...up to 8 LR patches
    QM000.png       # Quality mask per LR patch
    metadata.json   # Tile metadata

Only tiles with good status are exported. The exporter creates 128x128 normalized patches from the validated 384x384 tiles. This output is the direct input for PIUnet training.

Source: lwir_tile_validator/src/frontend/export/probav_exporter.py

Data Dependencies

All data is read-only from LWIR-Align:

Data Path Description
LA Mosaics flight_21051/*.tif Contrast-enhanced GeoTIFF reference mosaics
HA Images flight_21052/filtered_enhanced_undistorted/group{1-8}/*.png 675 undistorted thermal frames
Tile Metadata comprehensive_tile_metadata/tile_metadata_mosaic{A,B}_{384x384,192x192}.json Tile definitions with coverage info
Homographies group{N}/refined_to_mosaic/H_*_to_mosaic{A,B}.json Frame-to-mosaic transformations

The tool requires the superglue-env conda environment (shared with LWIR-Align).

Source: lwir_tile_validator/docs/REQUIREMENTS.md

Configurations

Config Mosaic Tile Size Description
mosaicA_384 A 384x384 Primary validation target (~126 valid tiles)
mosaicA_192 A 192x192 Higher tile count, smaller patches
mosaicB_384 B 384x384 Secondary mosaic (~84 valid tiles)
mosaicB_192 B 192x192 Secondary mosaic, smaller patches

Total across all configs: 1,364+ tiles, with 210 valid 384x384 tiles meeting the minimum HA image coverage threshold.

UI Layout

+-----------------------------------------------------------+
| Menu Bar: File | Navigation | View | Export               |
+-----------------------------------------------------------+
| Control Panel: [Config v] [< Tile 47/210 >] [Progress]   |
+----------------------------+------------------------------+
|                            |                              |
|  Tile Annotation Widget    |    HAPatchGrid (8-9 patches) |
|  - Tile preview            |    +------+------+------+    |
|  - Status buttons          |    | P1   | P2   | P3   |    |
|  - Notes field             |    +------+------+------+    |
|                            |    | P4   | P5   | P6   |    |
|                            |    +------+------+------+    |
|                            |    | P7   | P8   | P9   |    |
|                            |    +------+------+------+    |
+----------------------------+------------------------------+

Relationship to Pipeline

lwir-align (registration)
    |
    v  [tile metadata, homographies, images]
lwir-tile-validator (visual QC)
    |
    v  [ProbaV-format export of "good" tiles]
piunet (MFSR training)

The validator is the quality gate: it ensures that only well-aligned tile/patch pairs enter the training set. Without it, registration errors from LWIR-Align (especially from difficult groups like group 3) would propagate into the neural network as noisy training signal.

Browse Mode

A refactoring (commit a928231) added a dual-mode design: the original validation mode plus an export browse mode for reviewing and configuring exports. This involved a major UI refactor when the main window exceeded 1,000 lines (commit a2913ae).

Key Files

Path Purpose
run_validator.py Main launcher script
src/main.py Application entry point
src/models/tile_data.py Tile metadata structures and loader
src/models/annotation.py Annotation data model (3-level hierarchy)
src/services/tile_navigator.py Navigation controller
src/services/tile_image_manager.py Image loading with LRU cache
src/frontend/main_window.py Main application window
src/frontend/ha_patch_grid.py GPU-rendered patch grid
src/frontend/export/probav_exporter.py ProbaV dataset exporter
src/config.py Configurable path management
src/project_paths.py Centralized paths

Environment

  • Conda environment: superglue-env (shared with LWIR-Align)
  • Last active: November 2025
  • Location: /home/geoff/projects/ceres/superrez/lwir_tile_validator/
  • Code size: ~6,000 lines Python across 45+ files (~2,000 reused from Hephaestus)