Trackpoint Coordinate System

Purpose

Plant Tracer trackpoints must use a bottom-left coordinate origin in all user-facing and persisted data. A trackpoint at (0, 0) is the lower-left corner of the analysis frame. x increases to the right and y increases upward.

The movie image itself is not flipped. Canvas rendering, mouse dragging, and OpenCV tracking still operate in native image coordinates, where (0, 0) is the upper-left corner and y increases downward.

Coordinate Spaces

Plant Tracer uses two coordinate spaces:

trackpoint coordinates

The persisted and user-visible coordinate system. Origin is the lower-left corner of the analysis frame. These values are stored in DynamoDB and exported in CSV.

canvas coordinates

The browser canvas and OpenCV image coordinate system. Origin is the upper-left corner of the displayed analysis frame. These values are used only for drawing, hit detection, dragging, and optical-flow tracking.

The conversion uses the actual analysis-frame height:

canvas_x = trackpoint_x
canvas_y = frame_height - trackpoint_y

trackpoint_x = canvas_x
trackpoint_y = frame_height - canvas_y

OpenCV may calculate subpixel coordinates while tracking. Browser marker edits are displayed and saved as rounded integer pixel coordinates so the marker table, edited marker payload, and exported CSV agree. The top edge of a 480 pixel analysis frame has trackpoint y = 480; the bottom edge has trackpoint y = 0.

Frame Height Source

Conversion must use the height of the analysis frame shown to the user, not the raw uploaded movie height.

The analysis frame is the rotated and scaled frame produced by lambda-resize/src/resize_app/mpeg_jpeg_zip.py. The browser can use the canvas controller’s natural image height once the background frame has loaded. Lambda derives the conversion height from the first processed OpenCV frame using the same rotation and scaling path as tracking, then supplies that height to legacy migration if the movie row does not yet have stored dimensions.

DynamoDB Contract

The movies row owns the coordinate-system contract for all trackpoints in that movie:

trackpoint_origin: Literal["bottom-left"] | None = None
frame_height_px: int | None = None

Rules:

  • Missing or None trackpoint_origin means legacy "top-left" storage. The implementation must not write "top-left" to new movie rows.

  • trackpoint_origin = "bottom-left" means all stored frame trackpoints for that movie use the lower-left-origin contract.

  • New movies are created with trackpoint_origin = "bottom-left" in odb.create_new_movie(). POST /api/new-movie calls this function before returning the presigned upload form, and local tooling that creates movies should use the same function.

  • A movie must not contain mixed-origin frame trackpoints.

  • Once a movie’s stored frame trackpoints are converted, the movie row is updated to trackpoint_origin = "bottom-left".

  • Future trackpoint writes for a "bottom-left" movie store lower-left-origin values directly.

schema.Movie contains this field, and odb.TRACKPOINT_ORIGIN is the named string constant for the DynamoDB attribute.

The permanent coordinate contract belongs on the movie row, not on each frame or trackpoint. A per-frame migration marker (odb.TRACKPOINT_MIGRATION_ORIGIN) is internal idempotency machinery for lazy migration; it is not the public coordinate contract and is never exposed in API responses. The marker is durable (left in place after migration); once the movie row’s trackpoint_origin is "bottom-left" the migration routine returns early and never reads the markers again.

Legacy Migration

Existing movies without trackpoint_origin contain top-left-origin trackpoints. The selected migration strategy is server-side lazy migration of a complete movie before the first operation that exposes or writes trackpoints under the new contract.

The lazy migration trigger points are:

  • POST /api/get-movie-metadata when the request returns frame markers.

  • POST /api/get-movie-trackpoints before CSV or JSON export.

  • POST /api/put-frame-trackpoints before accepting edited markers.

  • Lambda retracing before reading existing seed trackpoints for optical flow.

Partial migration is not allowed. Saving one edited frame as bottom-left while other frames remain top-left would corrupt the movie’s trackpoint sequence. Because a movie can have more frame records than DynamoDB can update in one transaction, lazy migration must not be a naive in-place batch rewrite.

Migration must be resumable or fail closed:

  • Determine the analysis-frame height before any write. If it cannot be determined, return an error and leave the movie unchanged.

  • Convert each frame with an atomic conditional write that only flips when the frame is not already marked bottom-left (ConditionExpression='attribute_not_exists(#origin)'). This makes the per-frame flip idempotent, so a retry — or a concurrent migration of the same movie — can never double-flip a frame (a failed condition is caught and the existing result is left intact). See #1058.

  • Set trackpoint_origin = "bottom-left" only after every frame with trackpoints has been converted.

  • While a movie is in an incomplete migration state, editing, retracing, and exporting trackpoints must wait, retry, or return an error rather than expose mixed-origin data.

Browser Responsibilities

canvas_tracer_controller.mjs is responsible for translating between stored trackpoints and canvas objects.

When loading frame markers:

  • Read metadata.trackpoint_origin from the metadata object returned by POST /api/get-movie-metadata.

  • For "bottom-left" movies, convert stored trackpoints to canvas coordinates before creating Marker and Line objects.

  • Use metadata.frame_height_px for the Y conversion when supplied. This is the analysis coordinate height after rotation and scaling; do not rotate it again. With older responses, fall back to the loaded image’s natural height. Movie width/height can describe source dimensions rather than analysis dimensions (for example, a 480x360 source becomes 640x480).

  • Rebuild the current frame after the loaded image reports its natural dimensions. Do not flip bottom-left trackpoints against the source-movie or placeholder canvas height.

  • For legacy "top-left" movies that have not yet been migrated, use stored coordinates as canvas coordinates for visual correctness.

When dragging markers:

  • Keep the marker object in canvas coordinates so the marker follows the mouse over the unflipped movie.

  • Clamp dragged marker centers to the analysis-frame bounds so marker coordinates cannot leave the region of interest.

  • The marker table must display converted trackpoint coordinates live while the user drags.

  • get_markers() must return rounded integer trackpoint coordinates, not raw canvas coordinates, before posting to /api/put-frame-trackpoints.

The marker table and the posted payload must use the same conversion path so initial render, drag updates, saved data, and CSV export agree.

Trackpoints saved by a buggy client that flipped against the source-movie height remain numerically displaced. They cannot be migrated globally because the stored record does not identify which frame height the client used. Repair an affected frame 0 from evidence, then retrace it.

The Location (mm) column is calibrated only after both default ruler markers (Ruler 0mm and Ruler 10mm) have moved away from their starting positions. While a non-ruler marker is dragged after calibration, the pixel and millimeter columns update together in real time. While a ruler marker itself is being dragged, millimeter locations are withheld until the marker is released and the calibration can be recomputed from the new ruler distance.

Graph Responsibilities

Position graphs should consume trackpoint coordinates. Once frame data is bottom-left-origin, Y deltas are already positive upward. Chart.js should not reverse the Y axis to simulate a lower-left origin.

Flask API Responsibilities

POST /api/put-frame-trackpoints receives trackpoint coordinates. For legacy movies, Flask first runs the lazy migration. For "bottom-left" movies, Flask stores those values unchanged.

POST /api/get-movie-metadata returns the movie-row trackpoint_origin inside metadata.trackpoint_origin. The endpoint already returns the dictionary from odb.get_movie_metadata() inside the metadata response key, so the field is exposed once it exists on schema.Movie and is present in the movie row.

When POST /api/get-movie-metadata returns frame marker coordinates, Flask first runs the lazy migration if metadata.trackpoint_origin is missing or None. Returned frame markers therefore use the movie’s stored coordinate contract, which is bottom-left after migration.

POST /api/get-movie-trackpoints first runs the lazy migration and then exports stored trackpoint values directly. The CSV and JSON exports are therefore automatically lower-left origin without a separate export-only flip.

Lambda Tracking Responsibilities

OpenCV must receive canvas/image coordinates. Lambda tracking therefore converts bottom-left trackpoints to top-left image coordinates before calling optical flow or drawing labels, then converts tracer output back to bottom-left before writing frame trackpoints to DynamoDB.

The conversion uses the processed frame height derived from mpeg_jpeg_zip.get_first_frame_from_url(...).shape[0] rather than requiring the movie row to already have height. This keeps initial tracing from crashing when uploaded movie metadata has not yet been populated. The conversion formula is:

  • before cv2.calcOpticalFlowPyrLK: image_y = frame_height - trackpoint_y

  • before put_frame_trackpoints: trackpoint_y = frame_height - image_y

OpenCV reports a status for each marker independently. When the status says a marker could not be resolved, Lambda carries its last known position into the next frame. This preserves the marker for later frames and future retracing; successfully resolved markers continue to use the updated optical-flow position.

The traced MP4 and frame ZIP are visual artifacts. Marker overlays in those artifacts are drawn in image coordinates after conversion; the underlying movie frames are never flipped.

Tests

Substantive tests for the implementation should cover:

  • JavaScript marker load: bottom-left stored y draws at the expected canvas y.

  • JavaScript drag update: the marker table displays bottom-left y while the marker object remains in canvas coordinates.

  • JavaScript save: get_markers() posts rounded bottom-left coordinates.

  • JavaScript path drawing: lines between frames convert both endpoints before drawing.

  • Graphing: Y deltas use bottom-left values and do not rely on reversed chart axes.

  • ODB creation: odb.create_new_movie() stores trackpoint_origin = "bottom-left" for a new movie.

  • Flask metadata: POST /api/get-movie-metadata exposes the movie-row field as metadata.trackpoint_origin.

  • Flask CSV export: for a bottom-left movie, exported CSV values match stored trackpoints after integer export formatting.

  • Lambda tracking: input bottom-left trackpoints are converted before optical flow and converted back before persistence.

  • Migration: a legacy top-left movie is converted completely and marked trackpoint_origin = "bottom-left".

  • Migration retry: a failed lazy migration cannot double-flip already converted frames or expose a mixed-origin movie.

Documentation Follow-up

When the implementation lands, update user-facing coordinate descriptions in docs/UserTutorial.rst and src/app/templates/analyze.html. Any affected screenshots under docs/tutorial_images/ should be flagged for user review rather than replaced automatically.

Persisted frame height and downloads

Rotation is chosen before upload. Once upload completion is recorded or processing begins, the API rejects rotation changes with HTTP 409 and leaves existing tracking intact. Processing stores source dimensions, measured frame_height_px, and completion state in one update. The processed movie geometry is immutable; changing orientation requires a new upload. Tracing measures the same fixed coordinate space.

Legacy JPEG/ZIP height recovery persists a missing height. Repeated identical measurements are accepted; a conflicting measurement is rejected as inconsistent stored data, not a retryable rotation request. Legacy rows with saved dimensions or frames cannot be rotated through either the rotation API or the shared metadata writer, even if their old status still says uploading. Source dimensions are read-only through the metadata API, including missing fields. Both current and legacy upload-completion markers close geometry editing in the shared writer and source initializer. The synchronous CLI initializer accepts only a fresh record without a source, source dimensions, or saved frames; it never purges prior source or coordinate data. Trackpoint writes, migration, and marker map creation or renaming are rejected during upload setup, before any height recovery or coordinate mutation, even when a legacy coordinate origin is already present. The marker-map companion item at frame -100 is not a saved coordinate frame and alone does not block rotation or source initialization. Migration retains its existing conditional per-frame updates and durable conversion markers, so interrupted migrations can resume without flipping a frame twice. There is no migration path between different movie geometries.

Metadata-only requests use stored height or source dimensions without JPEG/ZIP reads or cache writes. With both source dimensions available, fallback height applies the tracer’s scaling and rotation. A height-only legacy record retains its historical interpretation until analysis pixels are measured.

Upload retries repair missing height on legacy completed uploads. Tracing failures during measurement or migration follow normal failure-status and lock cleanup. Upload processing records a failure when decoded source dimensions conflict with saved dimensions, including when the analysis height is missing or agrees. It preserves saved dimensions, coordinates, and source bytes on retries.

Browser metadata and JSON trackpoint downloads include frame_height_px and trackpoint_origin in metadata. CSV repeats them as columns; XLSX includes them on the Metadata sheet. Height is always pixels, independent of calibrated position units. Unknown height is JSON null or an empty spreadsheet cell. The frame height describes the entire resized, rotated image and is independent of the selected analysis frame range.

MP4 player validation on a dev stack

New uploads on the new-movieplayer branch use the untraced MP4 immediately when processing completes. The original stays intact. The browser and tracker share the derivative’s pixel dimensions; the source rotation must not be applied again. The analysis derivative includes every source frame, regardless of trim. Legacy backfill and bulk artifact deletion are separate work.

The automated gates are make check, make frame-step-browser-test, and make analysis-mp4-browser-test. The desktop workflow runs both browser gates on Windows Chrome and macOS Chrome. The production analyzer is exercised with both the encoder output and an independent B-frame fixture. Local storage tests use DynamoDB Local and MinIO, including an actual browser upload followed by stepping, marker save, tracing, and stepping again.

After deploying this branch to a dev stack using the normal Makefile deployment workflow, test on Windows and macOS:

  1. Upload a 640 by 480 and a 480 by 640 movie, choosing rotation before upload.

  2. Open Analyze as soon as processing finishes. Step forward and backward before tracing; verify adjacent burned-in frame numbers and correct orientation.

  3. Select a later frame, place markers, and trace from it. Verify saved marker positions, frame indices, and height in the downloaded data.

  4. After tracing, step in both directions and inspect marker overlays. Change playback speed and reverse direction. Change the trim range and verify that the full movie remains navigable.

  5. Verify the source download is unchanged, an untraced MP4 exists, and no new ZIP object appears. The traced download contains marker overlays without burned-in frame-number labels.

The player keeps one decoded keyframe group at a time (at most 128 MiB) and limits compressed input to 256 MiB. Decoder failures and missing derivatives are visible in the analyzer. This validation does not claim Safari conformance; Safari requires its own engine check before being declared supported.