Trails Extracted Using Satellite Images

Select a start + destination on the map, describe the vehicle configuration, submit a job, and see the extracted trail on the map, the pipeline images and the LLM evaluation.

Google Mapslocation selection
Sentinel-2satellite imagery (Copernicus)
Geminimarker generation
SAM3segmentation (web API)
OpenCVskeletonization + debug images

Google Map

Not loaded

Tip: click “Set current” or “Set destination”, then click on the map to drop a pin — or click “Try an example route” above for a ready-made pair, then Submit job.

User Config

Sent to

Profile

A preset fills the fields below; editing any of them switches back to Custom.

Wheel type Average speed (value)

Average speed (unit) Driving skill (0–1)

Safety preferences (0–1) Horsepower

Engine capacity Torque

Drive type 0–100 acceleration

Maximum speed

API note: this page calls POST /jobs then polls GET /jobs/<job_id> until completion. Load job (or ?job_id=<id>) shows an existing job without re-running it (and follows it if it is still running). A failed or partial job still shows what it produced.

CVResult

CV result preview

Not loaded

When the job finishes, its vectorized image is shown here and the trail is drawn on the map.

LLM Comment

Not loaded

Score

—

statusnot loaded

{}

Submit a job to load its evaluation.

Credits

SatTrail started as a prototype built by a team of four — listed alphabetically by surname: Elisabeth Didrich, Leo Frey, Me(Yaxian), and one more teammate — during a two-day hackathon, Mercedes-Benz Tech Innovation Hackathon „Navigation: The Nexus“, in 2025, where it was one of the winning teams. After the hackathon I added a SAM3 segmentation step, so the extracted trail comes out cleaner.

Below is a breakdown of each module: map → Gemini mark → SAM3 segment → OpenCV vectorize → georeference → evaluate.

Technical details

Artifacts from job — (run a job above, then scroll).

1) Area → satellite image (Sentinel-2)

Download the least cloudy Sentinel-2 true-colour image of the chosen area (Copernicus Sentinel Hub WMS), draw the A / B markers and save it as jobs/<job_id>/images/imagery.png.

# stages/download/sentinel_download.py
bbox = square_bbox(wanted, config.SENTINEL_MIN_SIDE_KM)
raw = fetch_image_bytes(_wms_url(), _wms_params(bbox, date.today()), SERVICE_NAME)
image_out.write_bytes(draw_route_markers(raw, bbox, Equirectangular, pair))

Downloaded satellite image

Not loaded

2) Prompt Gemini to draw the trail

We ask Gemini to overlay a 1px yellow trace so later steps can isolate the path, then scale its answer back to the input size → images/marked.png.

Why yellow: almost nothing in a landscape has that colour. Forest is green, lakes are blue or black, rock, soil and buildings are grey or brown, so a bright yellow line stands out against every background.

# stages/mark/gemini_mark.py
client = genai.Client(
  http_options=HttpOptions(base_url=config.env(ENV_API_BASE_URL)),
  api_key=config.require_env(ENV_API_KEY),
)
response = client.models.generate_content(
  model=MODEL_ID,
  contents=[PROMPT_YELLOW_GUESS, image],
  config=GenerateContentConfig(temperature=MODEL_TEMPERATURE),
)
marked.resize(size, RESIZE_FILTER).save(image_out)  # model answers at its own resolution

Gemini marked (yellow trail)

Not loaded

3) Segment “yellow” consistently with SAM3

We send the marked image to SAM3 and redraw the returned polygons as clean yellow → images/segmented.png. If SAM3 fails, the Gemini image passes through unchanged.

Why repaint: the next step reads raw pixel RGB values, and Gemini's yellow drifts (soft edges, shading, tints from the terrain underneath). SAM3 finds the shape; we paint it back in exact (255,255,0), so every trail pixel passes the colour test.

# stages/segment/sam3_segment.py
response = requests.post(sam3.api_url, json=_request_body(img, sam3.prompt), timeout=REQUEST_TIMEOUT)
polygons = _parse_polygons(response.json())  # every mask of every prediction
for polygon in polygons:
  cv.fillPoly(img, [polygon], config.SEGMENT_FILLPOL_COLOR)
  cv.polylines(img, [polygon], True, config.SEGMENT_POLYLINE_COLOR, POLYLINE_THICKNESS)
# SAM3 fails? the marked image is copied through, so vectorize still runs

SAM3 segmented mask

Not loaded

4) OpenCV: skeletonize, trace A → B, sample ~50 points

OpenCV does not "see" a trail; it only compares numbers. A pixel counts as trail only if its RGB falls in a tight yellow window (R ≥ 240, G ≥ 240, B ≤ 100, |R − G| ≤ 40, set in .env), which is why step 3 repaints in exact yellow.

Yellow mask → skeleton → cheapest path from A to B along the skeleton → sample evenly → images/vectorized.png + data/trail_pixels.json.

The sampled points are drawn in purple because purple is yellow's complementary colour: the dots and lines stand out clearly on top of the yellow trail.

# stages/vectorize/opencv_vectorize.py
yellow = (r >= YELLOW_R_MIN) & (g >= YELLOW_G_MIN) & (b <= YELLOW_B_MAX) & (np.abs(r - g) <= YELLOW_RG_DIFF_MAX)
skeleton = skeletonize(yellow)
skeleton_rc = np.argwhere(skeleton)
start, end = (_snap_to_skeleton(skeleton_rc, p) for p in endpoints)  # pins A, B
cost = np.where(skeleton, ON_SKELETON_COST, OFF_SKELETON_COST)
path, _ = route_through_array(cost, start, end, fully_connected=True, geometric=True)
points = sample_evenly(path, NUM_POINTS)  # keeps both ends

CV result (points + overlay)

Not loaded

5) Georeference: pixels → map coordinates

Project the key points with the image's footprint (bbox + CRS), look up an elevation per point, and write one GeoJSON Feature → data/trail.geojson.

# stages/georeference/georeference.py
footprint = load_footprint(layout.path(FOOTPRINT))
points = pixel_points_from_json(read_json(layout.path(TRAIL_PIXELS)))
lonlat = projection_for(footprint.crs).to_lonlat(points, footprint.bbox, image_size(layout.path(IMAGERY)))
coords = [LatLon(lat, lon) for lon, lat in lonlat]
slope = get_path_slope_information(coords, get_elevations(coords))
write_json(geojson_out, trail_feature(lonlat, path_metrics(lonlat)))

Trail GeoJSON (excerpt)

Not loaded

6) Evaluate: score the trail for the vehicle

Summarise the trail (length, slope, estimated time at your speed) and ask the LLM for a score and a comment → data/evaluation.json.

# stages/evaluate/llm_evaluate.py
summary = path_summary(user_config, read_json(layout.path(TRAIL)))
prompt = template.format(
  word_limit=config.env_int(ENV_WORD_LIMIT, DEFAULT_WORD_LIMIT),
  user_config_json=json.dumps(user_config, ensure_ascii=False),
  path_json=json.dumps(summary, ensure_ascii=False),
)
response = client.models.generate_content(model=model_id, contents=prompt, config=...)
return json.loads(_strip_code_fences(raw_text))  # {score, comment}

Evaluation

Not loaded