Segmentation Algorithm Workflows

This notebook demonstrates IceFloeTracker.jl's segmentation algorithms with very basic examples of their use.

# Setup environment
using Pkg
Pkg.add(;name="IceFloeTracker", rev="main")
Pkg.add("Images")

     Cloning git-repo `https://github.com/WilhelmusLab/IceFloeTracker.jl.git`
    Updating git-repo `https://github.com/WilhelmusLab/IceFloeTracker.jl.git`
   Resolving package versions...
    Updating `~/work/IceFloeTracker.jl/IceFloeTracker.jl/docs/Project.toml`
  [04643c7a] ~ IceFloeTracker v0.9.0 `~/work/IceFloeTracker.jl/IceFloeTracker.jl` ⇒ v0.9.0 `https://github.com/WilhelmusLab/IceFloeTracker.jl.git#main`
    Updating `~/work/IceFloeTracker.jl/IceFloeTracker.jl/docs/Manifest.toml`
  [04643c7a] ~ IceFloeTracker v0.9.0 `~/work/IceFloeTracker.jl/IceFloeTracker.jl` ⇒ v0.9.0 `https://github.com/WilhelmusLab/IceFloeTracker.jl.git#main`
Precompiling project...
  12181.4 ms  ✓ IceFloeTracker
  1 dependency successfully precompiled in 15 seconds. 427 already precompiled.
  1 dependency precompiled but a different version is currently loaded. Restart julia to access the new version. Otherwise, loading dependents of this package may trigger further precompilation to work with the unexpected version.
   Resolving package versions...
  No Changes to `~/work/IceFloeTracker.jl/IceFloeTracker.jl/docs/Project.toml`
  No Changes to `~/work/IceFloeTracker.jl/IceFloeTracker.jl/docs/Manifest.toml`

# Load packages
using IceFloeTracker: LopezAcosta2019, LopezAcosta2019Tiling, Watkins2025GitHub
using Images: erode, segment_mean, labels_map, SegmentedImage, RGB, mosaicview

Load the images

Load the dataset from https://github.com/danielmwatkins/icefloevalidation_dataset using the Watkins2025GitHub data loader.

data_loader = Watkins2025GitHub(; ref="a451cd5e62a10309a9640fbbe6b32a236fcebc70")

Watkins2025GitHub("a451cd5e62a10309a9640fbbe6b32a236fcebc70", "https://github.com/danielmwatkins/ice_floe_validation_dataset/", "data/validation_dataset/validation_dataset.csv", "/tmp/Watkins2025")

The available data are listed in the metadata field:

first(data_loader().metadata, 10)

For the example, we choose a single case from Baffin Bay in May 2022.

dataset = data_loader(c-> c.case_number == 6 && c.satellite == "terra")
case = first(dataset)

ValidationDataCase("006-baffin_bay-100km-20220530-terra-250m", Dict{Symbol, Any}(:sea_ice_fraction => 1.0, :visible_floes => "yes", :visible_landfast_ice => "no", :region => "baffin_bay", :cloud_category_manual => "thin", :qa_reviewer => "emma", :center_x => -762500, :modis_cloud_error => "no", :start_date => Date("2022-05-30"), :satellite => "terra"…), ColorTypes.RGBA{FixedPointNumbers.N0f8}[RGBA{N0f8}(0.651,0.675,0.722,1.0) RGBA{N0f8}(0.608,0.635,0.675,1.0) … RGBA{N0f8}(0.831,0.835,0.812,1.0) RGBA{N0f8}(0.824,0.827,0.835,1.0); RGBA{N0f8}(0.71,0.741,0.784,1.0) RGBA{N0f8}(0.675,0.702,0.741,1.0) … RGBA{N0f8}(0.882,0.882,0.851,1.0) RGBA{N0f8}(0.835,0.835,0.835,1.0); … ; RGBA{N0f8}(0.816,0.831,0.843,1.0) RGBA{N0f8}(0.839,0.855,0.867,1.0) … RGBA{N0f8}(0.898,0.902,0.918,1.0) RGBA{N0f8}(0.773,0.776,0.792,1.0); RGBA{N0f8}(0.882,0.886,0.902,1.0) RGBA{N0f8}(0.867,0.871,0.886,1.0) … RGBA{N0f8}(0.839,0.843,0.851,1.0) RGBA{N0f8}(0.776,0.78,0.788,1.0)], ColorTypes.RGBA{FixedPointNumbers.N0f8}[RGBA{N0f8}(0.145,0.588,0.651,1.0) RGBA{N0f8}(0.031,0.631,0.651,1.0) … RGBA{N0f8}(0.024,0.906,0.914,1.0) RGBA{N0f8}(0.047,0.898,0.933,1.0); RGBA{N0f8}(0.196,0.639,0.694,1.0) RGBA{N0f8}(0.098,0.694,0.714,1.0) … RGBA{N0f8}(0.02,0.929,0.929,1.0) RGBA{N0f8}(0.008,0.898,0.933,1.0); … ; RGBA{N0f8}(0.125,0.82,0.859,1.0) RGBA{N0f8}(0.094,0.855,0.863,1.0) … RGBA{N0f8}(0.051,0.945,0.949,1.0) RGBA{N0f8}(0.02,0.851,0.855,1.0); RGBA{N0f8}(0.11,0.898,0.925,1.0) RGBA{N0f8}(0.075,0.878,0.898,1.0) … RGBA{N0f8}(0.0,0.91,0.906,1.0) RGBA{N0f8}(0.0,0.82,0.82,1.0)], ColorTypes.RGBA{FixedPointNumbers.N0f8}[RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0) … RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0); RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0) … RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0); … ; RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0) … RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0); RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0) … RGBA{N0f8}(0.0,0.0,0.0,0.0) RGBA{N0f8}(0.0,0.0,0.0,0.0)], ColorTypes.RGBA{FixedPointNumbers.N0f8}[RGBA{N0f8}(0.004,0.533,0.933,1.0) RGBA{N0f8}(0.004,0.533,0.933,1.0) … RGBA{N0f8}(0.4,0.016,0.467,1.0) RGBA{N0f8}(0.718,0.067,0.553,1.0); RGBA{N0f8}(0.004,0.533,0.933,1.0) RGBA{N0f8}(0.004,0.533,0.933,1.0) … RGBA{N0f8}(0.4,0.016,0.467,1.0) RGBA{N0f8}(0.718,0.067,0.553,1.0); … ; RGBA{N0f8}(0.4,0.016,0.467,1.0) RGBA{N0f8}(0.4,0.016,0.467,1.0) … RGBA{N0f8}(0.0,0.012,1.0,1.0) RGBA{N0f8}(0.4,0.0,0.467,1.0); RGBA{N0f8}(0.4,0.016,0.467,1.0) RGBA{N0f8}(0.4,0.016,0.467,1.0) … RGBA{N0f8}(0.0,0.012,1.0,1.0) RGBA{N0f8}(0.4,0.0,0.467,1.0)], Gray{FixedPointNumbers.N0f8}[Gray{N0f8}(0.004) Gray{N0f8}(0.004) … Gray{N0f8}(0.004) Gray{N0f8}(0.004); Gray{N0f8}(0.004) Gray{N0f8}(0.004) … Gray{N0f8}(0.004) Gray{N0f8}(0.004); … ; Gray{N0f8}(0.004) Gray{N0f8}(0.004) … Gray{N0f8}(0.004) Gray{N0f8}(0.004); Gray{N0f8}(0.004) Gray{N0f8}(0.004) … Gray{N0f8}(0.004) Gray{N0f8}(0.004)], Gray{FixedPointNumbers.N0f8}[Gray{N0f8}(0.012) Gray{N0f8}(0.012) … Gray{N0f8}(0.012) Gray{N0f8}(0.012); Gray{N0f8}(0.012) Gray{N0f8}(0.012) … Gray{N0f8}(0.012) Gray{N0f8}(0.012); … ; Gray{N0f8}(0.012) Gray{N0f8}(0.012) … Gray{N0f8}(0.012) Gray{N0f8}(0.012); Gray{N0f8}(0.012) Gray{N0f8}(0.012) … Gray{N0f8}(0.012) Gray{N0f8}(0.012)], Gray{Bool}[Gray{Bool}(false) Gray{Bool}(false) … Gray{Bool}(false) Gray{Bool}(false); Gray{Bool}(false) Gray{Bool}(false) … Gray{Bool}(false) Gray{Bool}(false); … ; Gray{Bool}(false) Gray{Bool}(false) … Gray{Bool}(false) Gray{Bool}(false); Gray{Bool}(false) Gray{Bool}(false) … Gray{Bool}(false) Gray{Bool}(false)], Segmented Image with:
  labels map: 400×400 Matrix{Int64}
  number of labels: 177, 176x9 CSV file
label │ area  │ convex_area │ centroid-0 │ centroid-1 │ perimeter
──────┼───────┼─────────────┼────────────┼────────────┼──────────
1     │ 109.0 │ 116.0       │ 6.14679    │ 386.853    │ 38.7279  
2     │ 256.0 │ 268.0       │ 11.957     │ 209.793    │ 61.2132  
3     │ 104.0 │ 115.0       │ 10.5769    │ 308.413    │ 38.3848  
4     │ 268.0 │ 285.0       │ 19.7239    │ 251.519    │ 66.8701  
5     │ 55.0  │ 56.0        │ 14.3273    │ 370.109    │ 24.7279  
6     │ 97.0  │ 100.0       │ 18.4227    │ 237.763    │ 35.0711  
7     │ 288.0 │ 303.0       │ 23.9375    │ 356.965    │ 62.0416  
8     │ 65.0  │ 71.0        │ 21.5231    │ 183.646    │ 30.4853  
9     │ 113.0 │ 117.0       │ 21.292     │ 140.575    │ 38.1421  
10    │ 342.0 │ 364.0       │ 30.9532    │ 168.52     │ 77.3553  
... with 166 more rows, and 3 more columns: axis_major_length, axis_minor_length, case_number)

The data include the true-color image:

truecolor = RGB.(case.modis_truecolor) # TODO: remove RGB cast

... a false-color image:

falsecolor = RGB.(case.modis_falsecolor) # TODO: remove RGB cast

... and a landmask, which in this particular case is empty:

landmask = RGB.(case.modis_landmask) # TODO: remove RGB cast

Run the segmentation algorithm

The segmentation algorithm is an object with parameters as follows:

segmentation_algorithm = LopezAcosta2019()

LopezAcosta2019(Bool[0 0 … 0 0; 0 0 … 0 0; … ; 0 0 … 0 0; 0 0 … 0 0])

If we wanted to modify the options, we could include those in the call above. See the documentation for LopezAcosta2019 for details. The default parameters are as follows:

dump(segmentation_algorithm)

LopezAcosta2019
  landmask_structuring_element: Array{Bool}((99, 99)) Bool[0 0 … 0 0; 0 0 … 0 0; … ; 0 0 … 0 0; 0 0 … 0 0]

Run the algorithm as follows:

segments = segmentation_algorithm(truecolor, falsecolor, landmask)

Segmented Image with:
  labels map: 400×400 Matrix{Int64}
  number of labels: 59

To show the results with each segment marked using its mean color:

map(i -> segment_mean(segments, i), labels_map(segments))

We can do the same with the falsecolor image:

# Get the labels_map
segments_falsecolor = SegmentedImage(falsecolor, labels_map(segments))
map(i -> segment_mean(segments_falsecolor, i), labels_map(segments_falsecolor))

Let's compare the segmented output to the manually validated labels:

man_labels = case.validated_binary_floes
outlines = man_labels .- erode(man_labels)
seg_vs = map(i -> segment_mean(segments, i), labels_map(segments))
mosaicview(truecolor, seg_vs .* (1 .- Float64.(outlines)), nrow=1)

Run the segmentation algorithm with tiling

The "tiling" version of the algorithm is an object:

segmentation_algorithm_with_tiling = LopezAcosta2019Tiling()

LopezAcosta2019Tiling((rblocks = 2, cblocks = 2), (prelim_threshold = 0.43137254901960786, band_7_threshold = 0.7843137254901961, band_2_threshold = 0.7450980392156863, ratio_lower = 0.0, ratio_offset = 0.0, ratio_upper = 0.75), (white_threshold = 25.5, entropy_threshold = 4, white_fraction_threshold = 0.4), (gamma = 1.5, gamma_factor = 1.3, gamma_threshold = 220), (se_disk1 = Bool[0 1 0; 1 1 1; 0 1 0], se_disk2 = Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0], se_disk4 = Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]), (radius = 10, amount = 2.0, factor = 255.0), (band_7_threshold = 0.0196078431372549, band_2_threshold = 0.9019607843137255, band_1_threshold = 0.9411764705882353, band_7_threshold_relaxed = 0.0392156862745098, band_1_threshold_relaxed = 0.7450980392156863, possible_ice_threshold = 0.29411764705882354, k = 3), (radius = 10, amount = 2, factor = 0.5), 0.1)

It has more configurable parameters. For details, see the documentation of LopezAcosta2019Tiling. The default parameters are as follows:

dump(segmentation_algorithm_with_tiling)

LopezAcosta2019Tiling
  tile_settings: @NamedTuple{rblocks::Int64, cblocks::Int64}
    rblocks: Int64 2
    cblocks: Int64 2
  cloud_mask_thresholds: @NamedTuple{prelim_threshold::Float64, band_7_threshold::Float64, band_2_threshold::Float64, ratio_lower::Float64, ratio_offset::Float64, ratio_upper::Float64}
    prelim_threshold: Float64 0.43137254901960786
    band_7_threshold: Float64 0.7843137254901961
    band_2_threshold: Float64 0.7450980392156863
    ratio_lower: Float64 0.0
    ratio_offset: Float64 0.0
    ratio_upper: Float64 0.75
  adapthisteq_params: @NamedTuple{white_threshold::Float64, entropy_threshold::Int64, white_fraction_threshold::Float64}
    white_threshold: Float64 25.5
    entropy_threshold: Int64 4
    white_fraction_threshold: Float64 0.4
  adjust_gamma_params: @NamedTuple{gamma::Float64, gamma_factor::Float64, gamma_threshold::Int64}
    gamma: Float64 1.5
    gamma_factor: Float64 1.3
    gamma_threshold: Int64 220
  structuring_elements: @NamedTuple{se_disk1::Matrix{Bool}, se_disk2::Matrix{Bool}, se_disk4::Matrix{Bool}}
    se_disk1: Array{Bool}((3, 3)) Bool[0 1 0; 1 1 1; 0 1 0]
    se_disk2: Array{Bool}((5, 5)) Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]
    se_disk4: Array{Bool}((7, 7)) Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]
  unsharp_mask_params: @NamedTuple{radius::Int64, amount::Float64, factor::Float64}
    radius: Int64 10
    amount: Float64 2.0
    factor: Float64 255.0
  ice_masks_params: @NamedTuple{band_7_threshold::Float64, band_2_threshold::Float64, band_1_threshold::Float64, band_7_threshold_relaxed::Float64, band_1_threshold_relaxed::Float64, possible_ice_threshold::Float64, k::Int64}
    band_7_threshold: Float64 0.0196078431372549
    band_2_threshold: Float64 0.9019607843137255
    band_1_threshold: Float64 0.9411764705882353
    band_7_threshold_relaxed: Float64 0.0392156862745098
    band_1_threshold_relaxed: Float64 0.7450980392156863
    possible_ice_threshold: Float64 0.29411764705882354
    k: Int64 3
  prelim_icemask_params: @NamedTuple{radius::Int64, amount::Int64, factor::Float64}
    radius: Int64 10
    amount: Int64 2
    factor: Float64 0.5
  brighten_factor: Float64 0.1

segments = segmentation_algorithm_with_tiling(truecolor, falsecolor, landmask)

Segmented Image with:
  labels map: 400×400 Matrix{Int64}
  number of labels: 348

To show the results with each segment marked using its mean color:

map(i -> segment_mean(segments, i), labels_map(segments))

With the falsecolor image:

# Get the labels_map
segments_falsecolor = SegmentedImage(falsecolor, labels_map(segments))
map(i -> segment_mean(segments_falsecolor, i), labels_map(segments_falsecolor))

Let's compare the segmented output to the manually validated labels:

man_labels = case.validated_binary_floes
outlines = man_labels .- erode(man_labels)
seg_vs = map(i -> segment_mean(segments, i), labels_map(segments))
mosaicview(truecolor, seg_vs .* (1 .- Float64.(outlines)), nrow=1)