Segmentation Algorithm Workflows

This notebook demonstrates IceFloeTracker.jl's segmentation algorithms with very basic examples of their use.

# Setup environment
using Pkg
Pkg.add(;name="IceFloeTracker", rev="main")
Pkg.add("Images")

# Load packages
using IceFloeTracker: LopezAcosta2019, LopezAcosta2019Tiling, Watkins2026Dataset, info, modis_truecolor, modis_landmask, modis_falsecolor, validated_binary_floes
using Images: erode, segment_mean, labels_map, SegmentedImage, RGB, mosaicview

Load the images

Load the dataset from https://github.com/danielmwatkins/icefloevalidation_dataset using the Watkins2026Dataset data loader.

full_dataset = Watkins2026Dataset(; ref="v0.1")

Dataset(GitHubLoader("https://github.com/danielmwatkins/ice_floe_validation_dataset/", "v0.1", "/tmp/Watkins2026"), 378×30 DataFrame
 Row │        case_number  region          start_date  center_lon  center_lat  ⋯
     │ Int64  Int64        String          Dates.Date  Float64     Float64     ⋯
─────┼──────────────────────────────────────────────────────────────────────────
   1 │     0            1  baffin_bay      2022-09-11    -91.5275     77.801   ⋯
   2 │     1            1  baffin_bay      2022-09-11    -91.5275     77.801
   3 │     2            2  baffin_bay      2015-03-12    -81.9643     76.0579
   4 │     3            2  baffin_bay      2015-03-12    -81.9643     76.0579
   5 │     4            3  baffin_bay      2012-04-19    -79.5793     75.6372  ⋯
   6 │     5            3  baffin_bay      2012-04-19    -79.5793     75.6372
   7 │     6            4  baffin_bay      2019-09-25    -76.8094     79.3029
   8 │     7            4  baffin_bay      2019-09-25    -76.8094     79.3029
  ⋮  │   ⋮         ⋮             ⋮             ⋮           ⋮           ⋮       ⋱
 372 │   371          186  sea_of_okhostk  2017-05-23    155.183      59.6011  ⋯
 373 │   372          187  sea_of_okhostk  2013-04-29    155.731      59.7278
 374 │   373          187  sea_of_okhostk  2013-04-29    155.731      59.7278
 375 │   374          188  sea_of_okhostk  2015-03-12    156.838      57.6608
 376 │   375          188  sea_of_okhostk  2015-03-12    156.838      57.6608  ⋯
 377 │   376          189  sea_of_okhostk  2012-04-26    163.086      61.4172
 378 │   377          189  sea_of_okhostk  2012-04-26    163.086      61.4172
                                                 24 columns and 363 rows omitted)

The available data can be viewed:

first(info(full_dataset), 10)

For the example, we choose a single case from Baffin Bay in May 2022.

dataset = filter(c-> c.case_number == 6 && c.satellite == "terra", full_dataset)
case = first(dataset)

Case(GitHubLoader("https://github.com/danielmwatkins/ice_floe_validation_dataset/", "v0.1", "/tmp/Watkins2026"), DataFrameRow
 Row │        case_number  region      start_date  center_lon  center_lat  cen ⋯
     │ Int64  Int64        String      Dates.Date  Float64     Float64     Int ⋯
─────┼──────────────────────────────────────────────────────────────────────────
   1 │    10            6  baffin_bay  2022-05-30    -73.3612     75.2608   -7 ⋯
                                                              24 columns omitted)

The data include the true-color image:

truecolor = modis_truecolor(case)

... a false-color image:

falsecolor = modis_falsecolor(case)

... and a landmask, which in this particular case is empty:

landmask = modis_landmask(case)

Run the segmentation algorithm

The segmentation algorithm is an object with parameters as follows:

segmentation_algorithm = LopezAcosta2019.Segment()

IceFloeTracker.LopezAcosta2019.Segment(Bool[0 0 … 0 0; 0 0 … 0 0; … ; 0 0 … 0 0; 0 0 … 0 0])

If we wanted to modify the options, we could include those in the call above. See the documentation for LopezAcosta2019.Segment for details. The default parameters are as follows:

dump(segmentation_algorithm)

IceFloeTracker.LopezAcosta2019.Segment
  landmask_structuring_element: Array{Bool}((99, 99)) Bool[0 0 … 0 0; 0 0 … 0 0; … ; 0 0 … 0 0; 0 0 … 0 0]

Run the algorithm as follows:

segments = segmentation_algorithm(RGB.(truecolor), RGB.(falsecolor), landmask)

Segmented Image with:
  labels map: 400×400 Matrix{Int64}
  number of labels: 59

To show the results with each segment marked using its mean color:

map(i -> segment_mean(segments, i), labels_map(segments))

We can do the same with the falsecolor image:

# Get the labels_map
segments_falsecolor = SegmentedImage(falsecolor, labels_map(segments))
map(i -> segment_mean(segments_falsecolor, i), labels_map(segments_falsecolor))

Let's compare the segmented output to the manually validated labels:

man_labels = validated_binary_floes(case)
outlines = man_labels .- erode(man_labels)
seg_vs = map(i -> segment_mean(segments, i), labels_map(segments))
mosaicview(truecolor, seg_vs .* (1 .- Float64.(outlines)), nrow=1)

Run the segmentation algorithm with tiling

The "tiling" version of the algorithm is an object:

segmentation_algorithm_with_tiling = LopezAcosta2019Tiling.Segment()

IceFloeTracker.LopezAcosta2019Tiling.Segment((rblocks = 2, cblocks = 2), (prelim_threshold = 0.43137254901960786, band_7_threshold = 0.7843137254901961, band_2_threshold = 0.7450980392156863, ratio_lower = 0.0, ratio_offset = 0.0, ratio_upper = 0.75), (white_threshold = 25.5, entropy_threshold = 4, white_fraction_threshold = 0.4), (gamma = 1.5, gamma_factor = 1.3, gamma_threshold = 220), (se_disk1 = Bool[0 1 0; 1 1 1; 0 1 0], se_disk2 = Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0], se_disk4 = Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]), (radius = 10, amount = 2.0, factor = 255.0), (band_7_max = 0.0196078431372549, band_2_min = 0.9019607843137255, band_1_min = 0.9411764705882353, band_7_max_relaxed = 0.0392156862745098, band_1_min_relaxed = 0.7450980392156863, possible_ice_threshold = 0.29411764705882354), (radius = 10, amount = 2, factor = 0.5), 0.1)

It has more configurable parameters. For details, see the documentation of LopezAcosta2019Tiling.Segment. The default parameters are as follows:

dump(segmentation_algorithm_with_tiling)

IceFloeTracker.LopezAcosta2019Tiling.Segment
  tile_settings: @NamedTuple{rblocks::Int64, cblocks::Int64}
    rblocks: Int64 2
    cblocks: Int64 2
  cloud_mask_thresholds: @NamedTuple{prelim_threshold::Float64, band_7_threshold::Float64, band_2_threshold::Float64, ratio_lower::Float64, ratio_offset::Float64, ratio_upper::Float64}
    prelim_threshold: Float64 0.43137254901960786
    band_7_threshold: Float64 0.7843137254901961
    band_2_threshold: Float64 0.7450980392156863
    ratio_lower: Float64 0.0
    ratio_offset: Float64 0.0
    ratio_upper: Float64 0.75
  adapthisteq_params: @NamedTuple{white_threshold::Float64, entropy_threshold::Int64, white_fraction_threshold::Float64}
    white_threshold: Float64 25.5
    entropy_threshold: Int64 4
    white_fraction_threshold: Float64 0.4
  adjust_gamma_params: @NamedTuple{gamma::Float64, gamma_factor::Float64, gamma_threshold::Int64}
    gamma: Float64 1.5
    gamma_factor: Float64 1.3
    gamma_threshold: Int64 220
  structuring_elements: @NamedTuple{se_disk1::Matrix{Bool}, se_disk2::Matrix{Bool}, se_disk4::Matrix{Bool}}
    se_disk1: Array{Bool}((3, 3)) Bool[0 1 0; 1 1 1; 0 1 0]
    se_disk2: Array{Bool}((5, 5)) Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]
    se_disk4: Array{Bool}((7, 7)) Bool[0 0 … 0 0; 0 1 … 1 0; … ; 0 1 … 1 0; 0 0 … 0 0]
  unsharp_mask_params: @NamedTuple{radius::Int64, amount::Float64, factor::Float64}
    radius: Int64 10
    amount: Float64 2.0
    factor: Float64 255.0
  ice_masks_params: @NamedTuple{band_7_max::Float64, band_2_min::Float64, band_1_min::Float64, band_7_max_relaxed::Float64, band_1_min_relaxed::Float64, possible_ice_threshold::Float64}
    band_7_max: Float64 0.0196078431372549
    band_2_min: Float64 0.9019607843137255
    band_1_min: Float64 0.9411764705882353
    band_7_max_relaxed: Float64 0.0392156862745098
    band_1_min_relaxed: Float64 0.7450980392156863
    possible_ice_threshold: Float64 0.29411764705882354
  prelim_icemask_params: @NamedTuple{radius::Int64, amount::Int64, factor::Float64}
    radius: Int64 10
    amount: Int64 2
    factor: Float64 0.5
  brighten_factor: Float64 0.1

segments = segmentation_algorithm_with_tiling(truecolor, falsecolor, landmask)

Segmented Image with:
  labels map: 400×400 Matrix{Int64}
  number of labels: 351

To show the results with each segment marked using its mean color:

map(i -> segment_mean(segments, i), labels_map(segments))

With the falsecolor image:

# Get the labels_map
segments_falsecolor = SegmentedImage(falsecolor, labels_map(segments))
map(i -> segment_mean(segments_falsecolor, i), labels_map(segments_falsecolor))

Let's compare the segmented output to the manually validated labels:

man_labels = validated_binary_floes(case)
outlines = man_labels .- erode(man_labels)
seg_vs = map(i -> segment_mean(segments, i), labels_map(segments))
mosaicview(truecolor, seg_vs .* (1 .- Float64.(outlines)), nrow=1)