Grains segmentation of an orientation map (3/3)

The segmentation code is in the module LaueTools/orientationclustering.py (OC below).

• two UB matrices are linked if they belong to the same or neighbouring map points and their misorientation < threshold; clusters (grains) are the connected components
• all grains of the .fit files are used (_g0, _g1, ... several grains per map point)
• UB matrices are orthonormalised (polar decomposition) and misorientations take into account the crystal symmetry (24 equivalent orientations for cubic)
• map convention: mapdimension = invmapdims = (nb rows = slow axis, nb cols = fast axis), image index = row * nb cols + col
• cluster ids are sorted by decreasing size (cluster 0 is the largest)

This notebook is part of the LaueTools package (https://github.com/BM32ESRF/lauetools)

Author: J.-S. Micha, Last revision: Sept 2026. Tested with python 3.12 (jupyterlab, jupyterhub, VS Code)

notebook	input	output
1_index_refine.ipynb	.cor files (peak search results)	.fit files (indexed spots, UB matrix, strain per grain)
2_postprocess_maps.ipynb	.fit files	maps & statistics: nb of spots, pixel deviation, strain, lattice parameters, orientation
3_segmentation_grains.ipynb	.fit files	grains (clusters of orientation), KAM, GROD, boundaries

All settings of an experiment are in a YAML configuration file: copy config_template.yaml to configs/my_experiment.yaml, edit it and set CONFIG in the first code cell. The same file is used by the 3 notebooks. configs/ contains examples of BM32 datasets (access to ESRF /data needed).

SETTINGS

CONFIG = 'configs/<proposal>_MgO.yaml'   # configuration file of your experiment

FIT_FOLDER = None       # None: output fit_folder of the configuration file
THRESHOLD = .8          # deg, max misorientation between neighbouring points of a grain
MIN_CLUSTER_SIZE = 15    # min nb of points of a grain

# Parameters (notebook_to_html)
CONFIG = 'configs/<proposal>_MgO.yaml'

IMPORTS

import time
from pathlib import Path

import numpy as np
import matplotlib.pyplot as plt

try:
    import ipympl  # interactive plots (zoom, hover) in JupyterLab / VS Code
    %matplotlib widget
except ImportError:
    %matplotlib inline

import LaueTools.indexing_batch as IB
import LaueTools.orientationclustering as OC

-- warning: sklearn.metrics is not installed. For matchingrate.py module, it could be useful... but not necessary
-- WARNING: module Image or PIL is not installed, but only used for templateimagematching
loading readmccd.py

-- WARNING: Missing library libtiff, Please install: pylibtiff if you need open some tiff images. However, Fabio or PIL can do the job!
hostname = <host>

LOAD CONFIGURATION

cfg = IB.load_config(CONFIG)
IB.print_config(cfg)
scan, post = cfg['scan'], cfg['postprocess']
mapdims, invmapdims = scan['mapdims'], scan['invmapdims']   # (fast, slow), (slow, fast)
fit_folder = Path(FIT_FOLDER) if FIT_FOLDER else cfg['output']['fit_folder']

fit_folder = '/data/visitor/<proposal>/bm32/<date>/PROCESSED_DATA/<sample>/<dataset>/scan0001/fitfiles_MgO_6grains'
print('.fit files in', fit_folder)

---- <proposal> MgO ----  (<inhouse path>)
.cor files:  /data/visitor/<proposal>/bm32/<date>/PROCESSED_DATA/<sample>/<dataset>/scan0001/corfiles/eiger4m_####.cor
.fit files:  /data/visitor/<proposal>/bm32/<date>/PROCESSED_DATA/<sample>/<dataset>/scan0001/fitfiles_MgO
map:         mapdims (fast, slow) = (51, 51), motors (yech, xech)
detector:    EIGER_4MCdTe, calibration /data/visitor/<proposal>/bm32/<date>/RAW_DATA/<sample>/<dataset>/scan0001/calibGe001_<sample>_RT_mardi.det
material:    MgO, energy 5-22 keV (indexing up to 19 keV), nbGrainstoFind = 3
.fit files in /data/visitor/<proposal>/bm32/<date>/PROCESSED_DATA/<sample>/<dataset>/scan0001/fitfiles_MgO_6grains

RUN segmentation

t0 = time.time()
result, analyzer = OC.analyze_ub_matrices(input_dir=fit_folder,
                                          threshold=THRESHOLD,          # max misorientation (deg) between neighbouring points
                                          mapdimension=invmapdims,      # (nb rows = slow axis, nb cols = fast axis)
                                          prefix=scan['prefix'],        # e.g. eiger4m_0123_g0.fit
                                          symmetry=post['symmetry'],    # 'auto': cubic if all materials are cubic, None: raw angles
                                          spatial_connectivity=True,
                                          connectivity_type='8',        # '4': edges only, '8': edges + corners
                                          min_cluster_size=MIN_CLUSTER_SIZE,
                                          check_symmetry=True)          # ALARM if matrices are equivalent by symmetry but not reduced
print(f'elapsed time {time.time() - t0:.2f} s')
print(f"Grain indices: {analyzer.get_all_grain_indices()}")
OC.print_cluster_summary(result, analyzer, nb_max=20)

WARNING: 16 pairs of neighbouring UB matrices look different (raw angle > 2.0°) but are equivalent by cubic symmetry (misorientation < 2.0°). Matrices are NOT reduced to a common representative: use symmetry-aware tools (this module) for any comparison.
    img 1621 g4 <-> img 1672 g4: raw 89.27°, reduced 1.055°
    img 1621 g4 <-> img 1673 g4: raw 89.37°, reduced 0.948°
    img 1622 g3 <-> img 1672 g4: raw 89.15°, reduced 1.505°
    img 1622 g3 <-> img 1673 g4: raw 89.26°, reduced 1.402°
    img 1623 g3 <-> img 1673 g4: raw 89.28°, reduced 1.398°
NOTE: 5 UB matrices far from a rotation (singular values off by > 5%): poor refinement?
    img 169 g4: singular values [1.003, 0.979, 0.947]
    img 203 g3: singular values [1.043, 0.99, 0.947]
    img 300 g3: singular values [1.051, 1.007, 0.923]
    img 347 g3: singular values [1.007, 0.978, 0.944]
    img 2490 g5: singular values [1.052, 1.023, 0.981]

WARNING: in 15016/15016 grain blocks of .fit files, crystal-frame strain and lattice parameters are expressed in a cubic setting different from the written UB matrix (UB written as 'lowest Euler angles' representative). Maps of crystal-frame strain or lattice parameters read directly from files (e.g. ffs0.exx, ffs0.a) mix settings. Sample-frame strain is not affected. Use matrix_properties() / cluster_property_map().

elapsed time 94.36 s
Grain indices: [0, 1, 2, 3, 4, 5]
ClusterResult(81 clusters, threshold=0.8°, symmetry=cubic, mode=all_connected_8): 13709/15016 matrices assigned
  Cluster   0:  4574 matrices,  2243 pixels, mean (row, col)=( 18.6,  25.0), ref img   994, std=0.87°, max dev=3.53°, grains [0, 1, 2, 3, 4, 5]
  Cluster   1:   784 matrices,   758 pixels, mean (row, col)=( 29.5,  18.1), ref img  1548, std=0.12°, max dev=0.94°, grains [0, 1, 2, 3, 4, 5]
  Cluster   2:   771 matrices,   764 pixels, mean (row, col)=( 37.5,  35.8), ref img  1923, std=0.09°, max dev=0.84°, grains [0, 1, 2, 3, 4, 5]
  Cluster   3:   556 matrices,   550 pixels, mean (row, col)=( 24.6,  17.2), ref img  1292, std=0.11°, max dev=0.68°, grains [0, 1, 2, 3, 4, 5]
  Cluster   4:   461 matrices,   407 pixels, mean (row, col)=( 11.8,  37.3), ref img   649, std=0.18°, max dev=0.99°, grains [0, 1, 2, 3, 4, 5]
  Cluster   5:   434 matrices,   433 pixels, mean (row, col)=( 14.8,  11.2), ref img   776, std=0.05°, max dev=0.71°, grains [0, 1, 2, 3, 4, 5]
  Cluster   6:   431 matrices,   428 pixels, mean (row, col)=( 41.2,  12.1), ref img  2103, std=0.05°, max dev=0.54°, grains [0, 1, 2, 3, 4, 5]
  Cluster   7:   383 matrices,   383 pixels, mean (row, col)=( 43.8,  16.4), ref img  2260, std=0.05°, max dev=0.48°, grains [0, 1, 2, 3, 4, 5]
  Cluster   8:   381 matrices,   379 pixels, mean (row, col)=( 39.2,  38.3), ref img  2027, std=0.09°, max dev=0.65°, grains [0, 1, 2, 3, 4, 5]
  Cluster   9:   354 matrices,   351 pixels, mean (row, col)=( 18.0,  20.3), ref img   938, std=0.10°, max dev=0.78°, grains [0, 1, 2, 3, 4, 5]
  Cluster  10:   351 matrices,   285 pixels, mean (row, col)=( 19.7,  38.4), ref img  1058, std=0.23°, max dev=1.04°, grains [0, 1, 2, 3, 4, 5]
  Cluster  11:   313 matrices,   302 pixels, mean (row, col)=( 43.1,  26.5), ref img  2219, std=0.16°, max dev=1.07°, grains [0, 1, 2, 3, 4, 5]
  Cluster  12:   203 matrices,   199 pixels, mean (row, col)=( 29.4,  18.2), ref img  1497, std=0.09°, max dev=0.52°, grains [0, 1, 2, 3, 4, 5]
  Cluster  13:   161 matrices,   145 pixels, mean (row, col)=(  3.1,   9.4), ref img   162, std=0.19°, max dev=0.85°, grains [0, 1, 2, 3, 4, 5]
  Cluster  14:   151 matrices,   151 pixels, mean (row, col)=( 46.8,  11.2), ref img  2408, std=0.09°, max dev=0.49°, grains [0, 1, 2, 3, 4, 5]
  Cluster  15:   140 matrices,   140 pixels, mean (row, col)=( 39.3,   5.4), ref img  1995, std=0.04°, max dev=0.30°, grains [0, 1, 2, 3, 4, 5]
  Cluster  16:   132 matrices,   127 pixels, mean (row, col)=( 47.3,  41.1), ref img  2438, std=0.12°, max dev=0.89°, grains [0, 1, 2, 3, 4, 5]
  Cluster  17:   129 matrices,   128 pixels, mean (row, col)=( 25.1,  46.8), ref img  1322, std=0.07°, max dev=0.45°, grains [0, 1, 2, 3, 4, 5]
  Cluster  18:   123 matrices,   118 pixels, mean (row, col)=( 36.5,  39.0), ref img  1926, std=0.22°, max dev=1.33°, grains [0, 1, 2, 3, 4, 5]
  Cluster  19:   121 matrices,   121 pixels, mean (row, col)=( 36.6,   6.8), ref img  1894, std=0.04°, max dev=0.23°, grains [0, 1, 2, 3, 4, 5]
  ... and 61 more clusters

symmetry check

check_symmetry_equivalents() compares matrices of the same and neighbouring map points. It warns when two matrices look very different (raw angle > angle_tol) but are equivalent by crystal symmetry, i.e. when the matrices of the .fit files have not been reduced to a common representative. The clustering above is symmetry-aware, so it is not affected, but any raw element-wise comparison (e.g. GT.getRotationAngleFrom2Matrices, maps of UB elements or Euler angles) is.

align_to_cluster_reference() returns the UB matrices transformed (UB.S) to the symmetry variant closest to the reference matrix of their cluster.

report = OC.check_symmetry_equivalents(analyzer, symmetry=result.symmetry, angle_tol=2.0, verbose=False)
print(f"{len(report['sym_equivalent_pairs'])} neighbouring pairs equivalent by symmetry only")
for i, j, raw_angle, reduced_angle in report['sym_equivalent_pairs'][:10]:
    print(f"  {analyzer.matrices[i]} <-> {analyzer.matrices[j]}: raw {raw_angle:.1f}°, reduced {reduced_angle:.2f}°")

# UB matrices brought to a common symmetry representative within each cluster, shape (nb matrices, 3, 3)
UB_aligned = OC.align_to_cluster_reference(result, analyzer)

16 neighbouring pairs equivalent by symmetry only
  UBMatrix(image=1621, grain=4, material=MgO, nb_indexed=19, file=eiger4m_1621_g4.fit) <-> UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit): raw 89.3°, reduced 1.05°
  UBMatrix(image=1621, grain=4, material=MgO, nb_indexed=19, file=eiger4m_1621_g4.fit) <-> UBMatrix(image=1673, grain=4, material=MgO, nb_indexed=25, file=eiger4m_1673_g4.fit): raw 89.4°, reduced 0.95°
  UBMatrix(image=1622, grain=3, material=MgO, nb_indexed=25, file=eiger4m_1622_g3.fit) <-> UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit): raw 89.2°, reduced 1.50°
  UBMatrix(image=1622, grain=3, material=MgO, nb_indexed=25, file=eiger4m_1622_g3.fit) <-> UBMatrix(image=1673, grain=4, material=MgO, nb_indexed=25, file=eiger4m_1673_g4.fit): raw 89.3°, reduced 1.40°
  UBMatrix(image=1623, grain=3, material=MgO, nb_indexed=22, file=eiger4m_1623_g3.fit) <-> UBMatrix(image=1673, grain=4, material=MgO, nb_indexed=25, file=eiger4m_1673_g4.fit): raw 89.3°, reduced 1.40°
  UBMatrix(image=1671, grain=4, material=MgO, nb_indexed=23, file=eiger4m_1671_g4.fit) <-> UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit): raw 89.4°, reduced 0.85°
  UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit) <-> UBMatrix(image=1672, grain=5, material=MgO, nb_indexed=20, file=eiger4m_1672_g5.fit): raw 89.1°, reduced 1.07°
  UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit) <-> UBMatrix(image=1673, grain=5, material=MgO, nb_indexed=20, file=eiger4m_1673_g5.fit): raw 89.1°, reduced 1.19°
  UBMatrix(image=1672, grain=5, material=MgO, nb_indexed=20, file=eiger4m_1672_g5.fit) <-> UBMatrix(image=1673, grain=4, material=MgO, nb_indexed=25, file=eiger4m_1673_g4.fit): raw 89.2°, reduced 0.95°
  UBMatrix(image=1672, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1672_g4.fit) <-> UBMatrix(image=1722, grain=4, material=MgO, nb_indexed=22, file=eiger4m_1722_g4.fit): raw 89.5°, reduced 1.02°

CLUSTERS statistics

# find cluster seen in a given sample pt (or image index)

# ClusterStats by cluster id (do not index the list returned by get_cluster_stats() with a cluster id)
stats_by_id = result.get_cluster_stats_dict(analyzer)

testimageindex = post['reference_image_index'] or cfg['run']['test_image_index']
for stats in result.get_cluster_stats_by_image(analyzer, image_index=testimageindex):
    print(f"Cluster {stats.cluster_id} contains image {testimageindex}")
    print(f"  {stats}")
    print(f"  reference UB matrix (image {stats.reference_image_index}): {stats.reference_matrix.tolist()}")
    print(f"  distribution of misorientations to reference: {OC.get_angular_deviation_distribution(stats)}")

Cluster 0 contains image 1300
  ClusterStats(id=0, size=4574, pixels=2243, ref_img=994, mean (row, col)=(18.6, 25.0), std_orient=0.87°, max_dev=3.53°)
  reference UB matrix (image 994): [[0.882764269, 0.401209427, 0.242725272], [-0.416111538, 0.907261973, 0.011538744], [-0.21545568, -0.11151946, 0.970026167]]
  distribution of misorientations to reference: {'mean': 1.721115774505197, 'std': 0.8659059595504122, 'min': 0.0, 'max': 3.5279859097367807, 'median': 1.8219880390990368, 'q25': 0.8929974660039807, 'q75': 2.3338036052190168}
Cluster 1 contains image 1300
  ClusterStats(id=1, size=784, pixels=758, ref_img=1548, mean (row, col)=(29.5, 18.1), std_orient=0.12°, max_dev=0.94°)
  reference UB matrix (image 1548): [[0.75014231, -0.618853808, 0.241006981], [0.630077199, 0.548838357, -0.549096339], [0.206663052, 0.562563881, 0.800506958]]
  distribution of misorientations to reference: {'mean': 0.11502495740578633, 'std': 0.11934967567371427, 'min': 1.7075472925031877e-06, 'max': 0.9367700693259812, 'median': 0.08616910975833011, 'q25': 0.05117674699671769, 'q75': 0.13140710284795165}
Cluster 3 contains image 1300
  ClusterStats(id=3, size=556, pixels=550, ref_img=1292, mean (row, col)=(24.6, 17.2), std_orient=0.11°, max_dev=0.68°)
  reference UB matrix (image 1292): [[0.871366775, -0.429199341, -0.239889374], [0.486343415, 0.685621889, 0.539354927], [-0.066644162, -0.585565759, 0.805993549]]
  distribution of misorientations to reference: {'mean': 0.10540181187150849, 'std': 0.10573781925218677, 'min': 0.0, 'max': 0.6847565480215619, 'median': 0.07984411632806424, 'q25': 0.05796604705314691, 'q75': 0.1031979953861847}
Cluster 9 contains image 1300
  ClusterStats(id=9, size=354, pixels=351, ref_img=938, mean (row, col)=(18.0, 20.3), std_orient=0.10°, max_dev=0.78°)
  reference UB matrix (image 938): [[0.773290122, 0.608669012, 0.182498096], [-0.293557629, 0.596475053, -0.745221719], [-0.561315014, 0.521028509, 0.638944268]]
  distribution of misorientations to reference: {'mean': 0.08235711793774952, 'std': 0.1014850967880246, 'min': 0.0, 'max': 0.7780644295826121, 'median': 0.046832116867145446, 'q25': 0.02831431200020911, 'q75': 0.0920630204707716}
Cluster 12 contains image 1300
  ClusterStats(id=12, size=203, pixels=199, ref_img=1497, mean (row, col)=(29.4, 18.2), std_orient=0.09°, max_dev=0.52°)
  reference UB matrix (image 1497): [[0.979071703, -0.030005724, -0.206744778], [0.140715133, 0.825365391, 0.546766833], [0.15403938, -0.562900161, 0.810924114]]
  distribution of misorientations to reference: {'mean': 0.0923796062943224, 'std': 0.08943158723305253, 'min': 0.0, 'max': 0.5153964216774422, 'median': 0.06257666407322919, 'q25': 0.04376945290602961, 'q75': 0.09981033295274758}

# reference UB matrix of each cluster (as written in the .fit file of the point closest to the cluster barycenter)
# printed as copy-paste ready lists (e.g. ub_matrices of the configuration file); which='mean': mean orientation
refs = OC.get_reference_matrices(result, analyzer, min_size=30, which='reference')
UBlist = [np.array(m) for m in refs.values()]

# cluster 0 (2243 pts, ref image 994)
[[0.882764269, 0.401209427, 0.242725272], [-0.416111538, 0.907261973, 0.011538744], [-0.215455680, -0.111519460, 0.970026167]]
# cluster 1 (758 pts, ref image 1548)
[[0.750142310, -0.618853808, 0.241006981], [0.630077199, 0.548838357, -0.549096339], [0.206663052, 0.562563881, 0.800506958]]
# cluster 2 (764 pts, ref image 1923)
[[0.717568127, 0.697547996, -0.019076902], [-0.445070297, 0.479193742, 0.755052809], [0.535308650, -0.532885657, 0.653463382]]
# cluster 3 (550 pts, ref image 1292)
[[0.871366775, -0.429199341, -0.239889374], [0.486343415, 0.685621889, 0.539354927], [-0.066644162, -0.585565759, 0.805993549]]
# cluster 4 (407 pts, ref image 649)
[[0.859177441, 0.417459475, 0.262274242], [-0.426609616, 0.898522026, -0.033853000], [-0.252510846, -0.084801770, 0.964371436]]
# cluster 5 (433 pts, ref image 776)
[[0.663612146, -0.423231371, -0.618504688], [0.475968340, 0.874246637, -0.088208160], [0.577286492, -0.235815535, 0.781165237]]
# cluster 6 (428 pts, ref image 2103)
[[0.669869057, -0.138881061, -0.729576958], [0.486396740, 0.823304691, 0.290472795], [0.559934769, -0.549064353, 0.619415822]]
# cluster 7 (383 pts, ref image 2260)
[[0.940743218, -0.213676647, -0.270871087], [0.280607934, 0.929499653, 0.239960242], [0.199652504, -0.301227141, 0.932229936]]
# cluster 8 (379 pts, ref image 2027)
[[0.650777042, -0.400058486, 0.646121497], [0.585118561, 0.805129347, -0.089955071], [-0.484426303, 0.436516235, 0.757851489]]
# cluster 9 (351 pts, ref image 938)
[[0.773290122, 0.608669012, 0.182498096], [-0.293557629, 0.596475053, -0.745221719], [-0.561315014, 0.521028509, 0.638944268]]
# cluster 10 (285 pts, ref image 1058)
[[0.985411590, -0.084519858, 0.147981257], [0.007747284, 0.888308477, 0.456540576], [-0.170194314, -0.448137931, 0.876280632]]
# cluster 11 (302 pts, ref image 2219)
[[0.700823404, -0.639021985, -0.312473449], [0.690270380, 0.717791343, 0.078985287], [0.173550676, -0.272434827, 0.946618855]]
# cluster 12 (199 pts, ref image 1497)
[[0.979071703, -0.030005724, -0.206744778], [0.140715133, 0.825365391, 0.546766833], [0.154039380, -0.562900161, 0.810924114]]
# cluster 13 (145 pts, ref image 162)
[[0.854685455, -0.490896495, -0.172128382], [0.218828206, 0.638585800, -0.736517207], [0.471304111, 0.591867324, 0.653644005]]
# cluster 14 (151 pts, ref image 2408)
[[0.643831704, 0.154142876, -0.747592383], [-0.506389556, 0.818599955, -0.270221154], [0.571873104, 0.552751097, 0.606513540]]
# cluster 15 (140 pts, ref image 1995)
[[0.738796758, 0.677166885, 0.071217100], [-0.474135560, 0.586445191, -0.661908344], [-0.488828860, 0.452536177, 0.746777423]]
# cluster 16 (127 pts, ref image 2438)
[[0.973762411, 0.228953133, 0.005199603], [-0.225984264, 0.959089992, 0.172336244], [0.035225952, -0.168761026, 0.985072420]]
# cluster 17 (128 pts, ref image 1322)
[[0.743392825, 0.606417436, -0.282692056], [-0.668666626, 0.683021570, -0.291778217], [0.015442128, 0.407425654, 0.913823914]]
# cluster 18 (118 pts, ref image 1926)
[[0.984072614, 0.139761421, 0.114802934], [-0.094416387, 0.946394447, -0.307828169], [-0.150948451, 0.286591733, 0.944454621]]
# cluster 19 (121 pts, ref image 1894)
[[0.722290642, 0.673957037, 0.151320009], [-0.667305937, 0.624809278, 0.406022415], [0.178625880, -0.394208529, 0.900949737]]
# cluster 20 (113 pts, ref image 1131)
[[0.544228841, 0.517207348, -0.667241978], [-0.694534789, 0.717673694, -0.013775759], [0.472376867, 0.462165498, 0.744723176]]
# cluster 21 (107 pts, ref image 2486)
[[0.712506880, 0.617324367, 0.335905409], [-0.681387118, 0.490115591, 0.543455353], [0.170962244, -0.615913368, 0.769135317]]
# cluster 22 (87 pts, ref image 615)
[[0.681230649, -0.288703929, 0.673263963], [0.620271860, 0.714144148, -0.320744345], [-0.387895270, 0.636194308, 0.665294720]]
# cluster 23 (107 pts, ref image 1827)
[[0.744237994, -0.608000203, 0.273441984], [0.601259200, 0.427776904, -0.676108076], [0.292488033, 0.669132842, 0.691973582]]
# cluster 24 (99 pts, ref image 802)
[[0.737979284, 0.606872238, 0.299647978], [-0.289571517, 0.683082836, -0.669837750], [-0.610454240, 0.406794553, 0.678270643]]
# cluster 25 (89 pts, ref image 2539)
[[0.535103725, -0.429242465, 0.732179819], [0.556967546, 0.825994110, 0.076726031], [-0.635343970, 0.364989115, 0.678818277]]
# cluster 26 (86 pts, ref image 1474)
[[0.732406275, -0.628651425, 0.260618735], [0.599718939, 0.419234340, -0.679501082], [0.318105408, 0.654894623, 0.684719503]]
# cluster 27 (85 pts, ref image 722)
[[0.651166857, -0.553761417, 0.524494365], [0.728460897, 0.655722387, -0.206635106], [-0.230218727, 0.513176886, 0.827589995]]
# cluster 28 (84 pts, ref image 2403)
[[0.963425751, -0.259075968, -0.072523260], [0.256141590, 0.965331470, -0.036200638], [0.078670112, 0.016557478, 0.996673405]]
# cluster 29 (74 pts, ref image 1788)
[[0.728024934, -0.677106294, 0.135186498], [0.612404051, 0.544637882, -0.574678231], [0.314721554, 0.499682162, 0.807382487]]
# cluster 30 (70 pts, ref image 2057)
[[0.944145384, 0.310845191, -0.104066472], [-0.297791592, 0.679264040, -0.670326937], [-0.139289417, 0.667811168, 0.733223691]]
# cluster 31 (70 pts, ref image 2362)
[[0.761363897, -0.329846857, -0.559230927], [0.426606870, 0.901877205, 0.048659450], [0.488579003, -0.276261115, 0.827551617]]
# cluster 32 (68 pts, ref image 2202)
[[0.695246996, -0.691346946, -0.169246128], [0.473131523, 0.628390468, -0.614575170], [0.536010774, 0.350578645, 0.770430449]]
# cluster 33 (61 pts, ref image 1577)
[[0.878450937, 0.435111118, 0.202562256], [-0.441233019, 0.896685158, -0.009935821], [-0.184684178, -0.080515816, 0.978768593]]
# cluster 34 (63 pts, ref image 2401)
[[0.784890188, 0.554333823, -0.275080743], [-0.619133176, 0.716949413, -0.320598762], [0.018992423, 0.422883379, 0.906305430]]
# cluster 35 (60 pts, ref image 692)
[[0.699601826, -0.682618920, -0.211031198], [0.419038949, 0.629197325, -0.652883621], [0.577828353, 0.366854206, 0.727920409]]
# cluster 36 (60 pts, ref image 1278)
[[0.975845693, -0.218469694, 0.021564899], [0.217402070, 0.953073208, -0.203490808], [0.023506574, 0.202970738, 0.976569487]]
# cluster 37 (53 pts, ref image 1227)
[[0.974456286, -0.146924016, -0.169562551], [0.214054695, 0.836510000, 0.501369005], [0.068048547, -0.525397102, 0.846717391]]
# cluster 38 (51 pts, ref image 751)
[[0.863197666, 0.148308506, -0.485863344], [-0.342507079, 0.874998757, -0.341329877], [0.373917254, 0.460020026, 0.804637837]]
# cluster 39 (51 pts, ref image 1379)
[[0.889599536, -0.442761094, -0.125646955], [0.236289058, 0.673543648, -0.700566350], [0.394461050, 0.592564413, 0.702407499]]
# cluster 40 (51 pts, ref image 1469)
[[0.908702867, 0.398226942, 0.145274907], [-0.382431019, 0.916807589, -0.118537037], [-0.179648865, 0.053385014, 0.982212314]]
# cluster 41 (50 pts, ref image 525)
[[0.866787952, 0.418445707, 0.267214918], [-0.428214737, 0.905147348, -0.031361728], [-0.254349307, -0.087294140, 0.965564934]]
# cluster 42 (47 pts, ref image 93)
[[0.909449858, 0.345934825, 0.234301673], [-0.359929508, 0.929850114, 0.024458860], [-0.208742430, -0.106759161, 0.971816701]]
# cluster 43 (42 pts, ref image 2011)
[[0.801025836, -0.538175002, 0.258622733], [0.458193378, 0.828597685, 0.313328898], [-0.385075643, -0.135693068, 0.907786561]]
# cluster 44 (45 pts, ref image 1978)
[[0.989238768, 0.126749518, 0.074963545], [-0.095870877, 0.938748809, -0.328176174], [-0.111216822, 0.317664901, 0.941674958]]
# cluster 45 (42 pts, ref image 258)
[[0.935869527, 0.343230817, 0.087606721], [-0.347786310, 0.933431613, 0.076313451], [-0.054873699, -0.102535385, 0.992943214]]
# cluster 46 (42 pts, ref image 2087)
[[0.833859209, 0.284130612, 0.472398574], [-0.003104337, 0.860723668, -0.510252938], [-0.551998260, 0.423112919, 0.719380779]]
# cluster 47 (39 pts, ref image 976)
[[0.959394453, -0.166618068, -0.237816197], [0.143288020, 0.983835079, -0.110693872], [0.251866445, 0.072383556, 0.964941470]]
# cluster 48 (39 pts, ref image 2297)
[[0.974616510, -0.056222389, -0.227508595], [0.156025592, 0.879894932, 0.449375473], [0.174141714, -0.472827980, 0.864411662]]
# cluster 49 (38 pts, ref image 1703)
[[0.798841579, -0.517855361, 0.306953666], [0.460234362, 0.852533425, 0.241427676], [-0.388057278, -0.053336487, 0.919530237]]
# cluster 50 (32 pts, ref image 2409)
[[0.750286487, -0.642265638, -0.156579846], [0.622574662, 0.607151474, 0.493656006], [-0.221856894, -0.468175774, 0.856278848]]
# cluster 51 (34 pts, ref image 2319)
[[0.802303292, -0.511644811, 0.309868400], [0.459478909, 0.858332506, 0.227879110], [-0.382351915, -0.040472888, 0.923090888]]
# cluster 52 (32 pts, ref image 463)
[[0.903016103, 0.370062988, 0.214328873], [-0.375901474, 0.923811707, -0.010895749], [-0.201824978, -0.070840523, 0.976652529]]
# cluster 53 (32 pts, ref image 533)
[[0.948016898, 0.276985330, -0.161547859], [-0.298674186, 0.940574548, -0.152974712], [0.109723589, 0.191911135, 0.972045232]]
# cluster 54 (32 pts, ref image 143)
[[0.918529980, 0.335681239, 0.211446708], [-0.349565384, 0.937771647, -0.001140213], [-0.191520047, -0.077933651, 0.974724289]]
# cluster 55 (31 pts, ref image 536)
[[0.879568236, 0.409314241, 0.250739866], [-0.427513576, 0.903197552, 0.023913290], [-0.217185094, -0.127576924, 0.969532258]]
# cluster 56 (30 pts, ref image 2186)
[[0.860552134, 0.419189072, 0.263575139], [-0.428197767, 0.899230437, -0.031991747], [-0.253784629, -0.084529115, 0.964088291]]

# clusters sorted by orientation spread
for stats in result.get_cluster_stats_sorted(analyzer, sort_by='max_angular_deviation')[:15]:
    row, col = stats.mean_position
    print(f"Cluster {stats.cluster_id:3d}: {stats.nb_pixels:4d} pts, mean (col, row)=({col:.1f}, {row:.1f}), "
          f"std={stats.std_orientation:.2f}°, max misorientation={stats.max_angular_deviation:.2f}°")

Cluster   0: 2243 pts, mean (col, row)=(25.0, 18.6), std=0.87°, max misorientation=3.53°
Cluster  43:   42 pts, mean (col, row)=(22.3, 38.5), std=0.33°, max misorientation=1.51°
Cluster  18:  118 pts, mean (col, row)=(39.0, 36.5), std=0.22°, max misorientation=1.33°
Cluster  42:   47 pts, mean (col, row)=(42.2, 1.0), std=0.18°, max misorientation=1.08°
Cluster  11:  302 pts, mean (col, row)=(26.5, 43.1), std=0.16°, max misorientation=1.07°
Cluster  10:  285 pts, mean (col, row)=(38.4, 19.7), std=0.23°, max misorientation=1.04°
Cluster  49:   38 pts, mean (col, row)=(19.3, 33.0), std=0.22°, max misorientation=1.03°
Cluster  50:   32 pts, mean (col, row)=(12.1, 47.1), std=0.36°, max misorientation=1.03°
Cluster  22:   87 pts, mean (col, row)=(3.3, 12.3), std=0.27°, max misorientation=1.03°
Cluster  21:  107 pts, mean (col, row)=(38.5, 48.4), std=0.17°, max misorientation=1.00°
Cluster   4:  407 pts, mean (col, row)=(37.3, 11.8), std=0.18°, max misorientation=0.99°
Cluster  61:   25 pts, mean (col, row)=(18.3, 48.5), std=0.33°, max misorientation=0.99°
Cluster  77:   15 pts, mean (col, row)=(9.2, 48.6), std=0.32°, max misorientation=0.97°
Cluster  32:   68 pts, mean (col, row)=(9.5, 43.1), std=0.11°, max misorientation=0.97°
Cluster   1:  758 pts, mean (col, row)=(18.1, 29.5), std=0.12°, max misorientation=0.94°

# clusters built with spatial connectivity are connected by construction.
# split_non_compact_clusters() is useful after clustering without spatial connectivity
# (max_gap=1: points touching by edge or corner are connected, max_gap=2 bridges 1-point gaps)
split_result = OC.split_non_compact_clusters(result, analyzer, max_gap=1, min_cluster_size=MIN_CLUSTER_SIZE)
print(split_result)
print('cluster 0 compact:', OC.is_cluster_compact(result.get_cluster(0), analyzer))

ClusterResult(81 clusters, threshold=0.8°, symmetry=cubic, mode=all_connected_8_split)
cluster 0 compact: True

MAPS of clusters

all clusters

At map points where several grains are indexed (g0, g1, g2 ...), the grain with the largest number of indexed spots is shown. Black lines: boundaries between clusters. Hover the map to read image index and cluster id.

fig, ax = OC.plot_cluster_map(result, analyzer, min_size=MIN_CLUSTER_SIZE, show_ids=True, show_boundaries=True)

each individual cluster

Each cluster is colored by the misorientation (deg) of its points to the cluster reference matrix (red star: point closest to the cluster barycenter). Gray: other measured points.

# all clusters (largest first). vmax=None: each panel has its own color scale, vmax=1: common scale
# live=False (default): shown as a PNG image (light for jupyterlab); live=True: interactive plot (zoom, hover values)
fig = OC.plot_clusters_gallery(result, analyzer, min_size=30, max_clusters=8, nb_cols=4, vmax=None, zoom=False, live=False)

# browse clusters one by one: click in the list, then use the up/down arrow keys (or the prev/next buttons)
# property menu: GROD, KAM, Euler angles, strain (sample or crystal frame), lattice parameters, nb of spots, pixel deviation
# each value is read from the .fit file (and grain block) of the matrix of the cluster at each point
# 'crystal axes of cluster ref.': crystal-frame quantities expressed in the same cubic setting for all points
# 'live plot' checkbox: unchecked (default) maps shown as PNG images (fast, light); checked: interactive canvas
# the reference UB matrix of the selected cluster is shown as a copyable list (click in the box, Ctrl+A, Ctrl+C)
OC.visualize_cluster_interactive(result, analyzer, min_size=MIN_CLUSTER_SIZE, zoom=False, properties=True)

[interactive browser (ipywidgets): static view of the first cluster with default settings]

fig, ax = OC.plot_single_cluster(result, analyzer, cluster_id=2, zoom=False)

ClusterStats(id=2, size=771, pixels=764, ref_img=1923, mean (row, col)=(37.5, 35.8), std_orient=0.09°, max_dev=0.84°)
reference UB matrix: [[0.717568, 0.697548, -0.019077], [-0.44507, 0.479194, 0.755053], [0.535309, -0.532886, 0.653463]]

properties of a cluster (strain, lattice parameters, Euler angles)

Values are read, for each point, from the _g#.fit file (and grain block) of the matrix belonging to the cluster at this point.

Cubic materials: crystal-frame quantities (crystal strain, lattice parameters, Euler angles) are brought to the axes of the cluster reference matrix (aligned=True), otherwise they may jump from point to point between equivalent cubic settings:

• crystal-frame strain: recomputed from the sample-frame strain (setting independent) in the axes of the aligned UB
• lattice parameters: written values transformed to the aligned axes (a keeps its fixed value of the deviatoric refinement)
• Euler angles: LaueTools convention (orientations.calc_Euler_angles) of the aligned UB

Properties: OC.PROPERTY_GROUPS

print(OC.PROPERTY_GROUPS)
fig, ax = OC.plot_cluster_property(result, analyzer, cluster_id=1, prop='ezz_sample', aligned=True,
                                   zoom=False, symmetric=False, percentiles=(2, 98), show_envelope=True)

# values as 2D array (NaN outside the cluster, strain in absolute units)
exz_c0 = OC.cluster_property_map(result, analyzer, cluster_id=0, prop='exz', aligned=True)

{'orientation': ['grod', 'phi', 'theta', 'psi'], 'strain sample frame': ['exx_sample', 'eyy_sample', 'ezz_sample', 'exy_sample', 'exz_sample', 'eyz_sample'], 'strain crystal frame': ['exx', 'eyy', 'ezz', 'exy', 'exz', 'eyz'], 'lattice parameters': ['a', 'b', 'c', 'alpha', 'beta', 'gamma'], 'quality': ['nb_indexed', 'pixdev']}
ClusterStats(id=1, size=784, pixels=758, ref_img=1548, mean (row, col)=(29.5, 18.1), std_orient=0.12°, max_dev=0.94°)

# whole map: property of the primary grain of each point, crystal-frame quantities in the axes of the reference of its cluster
settings = OC.check_crystal_frame_settings(analyzer)
prop = 'b'
bmap = OC.property_map(result, analyzer, prop, aligned=True)
fig, ax = OC.plot_kam(bmap, result, analyzer, cmap='viridis',
                      vmin=np.nanpercentile(bmap, 1), vmax=np.nanpercentile(bmap, 99),
                      title=OC.property_label(prop) + ' (crystal axes of cluster references)')

WARNING: in 15016/15016 grain blocks of .fit files, crystal-frame strain and lattice parameters are expressed in a cubic setting different from the written UB matrix (UB written as 'lowest Euler angles' representative). Maps of crystal-frame strain or lattice parameters read directly from files (e.g. ffs0.exx, ffs0.a) mix settings. Sample-frame strain is not affected. Use matrix_properties() / cluster_property_map().

KAM and GROD

Kernel Average Misorientation: for each map point, mean misorientation between its orientation and those of its neighbours (connectivity='8': 8 first neighbours, or an integer d for all points within distance d). Neighbours with misorientation > max_angle (grain boundaries) or belonging to another cluster are ignored. KAM reveals local lattice curvature, i.e. geometrically necessary dislocations.

GROD (Grain Reference Orientation Deviation): misorientation of each point to the reference orientation of its cluster, i.e. long-range orientation gradients inside grains.

kam = OC.compute_kam(analyzer, result, connectivity='8', max_angle=2.0)
print(f'KAM mean {np.nanmean(kam):.3f}°, median {np.nanmedian(kam):.3f}°')
fig, ax = OC.plot_kam(kam, result, analyzer, vmax=None, show_boundaries=False)

KAM mean 0.145°, median 0.043°

grod = OC.compute_grod_map(result, analyzer)
fig, ax = OC.plot_kam(grod, result, analyzer, cmap='viridis', show_boundaries=True,
                      title='GROD: misorientation to cluster reference (deg)')

GROD and KAM of a single cluster

Only the orientations of the selected cluster are used, at all its points (also where it is not the primary grain of the point). Red line: envelope of the grain.

GROD reference: None = point closest to the cluster barycenter (red star), 'mean' = mean orientation of the cluster, an image index of the cluster, or any UB matrix.

cluster_id = 1
fig = OC.plot_cluster_grod_kam(result, analyzer, cluster_id=cluster_id,
                               reference=None,          # None, 'mean', image index, or UB matrix
                               zoom=False, show_envelope=True,
                               kam_max_angle=2.0, kam_connectivity='8',
                               vmax_grod=None, vmax_kam=None)

# maps as arrays (NaN outside the cluster)
grod_c = OC.compute_grod_map(result, analyzer, cluster_id=cluster_id, reference='mean')
kam_c = OC.compute_kam(analyzer, result, cluster_id=cluster_id)
plt.show()

ClusterStats(id=1, size=784, pixels=758, ref_img=1548, mean (row, col)=(29.5, 18.1), std_orient=0.12°, max_dev=0.94°)
reference UB matrix: [[0.750142, -0.618854, 0.241007], [0.630077, 0.548838, -0.549096], [0.206663, 0.562564, 0.800507]]

# browse clusters (click in the list, then up/down arrow keys) and choose the GROD reference
OC.cluster_grod_interactive(result, analyzer, min_size=MIN_CLUSTER_SIZE, zoom=False, show_envelope=True)

[interactive browser (ipywidgets): static view of the first cluster with default settings]

CLUSTER boundaries and envelopes

Even large clusters are sparse (points of other grains in between, several grains per image). The envelope of a grain is the region it occupies:

• method='closing' (default): morphological closing with a disk of radius map steps + hole filling. Bridges gaps up to ~2*radius and follows concave shapes. Envelope pieces with less than 3 points (stray points) are removed
• method='hull': convex hull, i.e. extreme limits of the grain (overestimates concave grains)
• method='bbox': bounding box

fill = fraction of the envelope actually occupied by the cluster points.

# main clusters superimposed: envelope outline + light filling + points (slightly shifted per cluster
# so that grains superimposed at the same point remain visible)
fig, ax, envelopes = OC.plot_cluster_envelopes(result, analyzer, min_size=100, max_clusters=10,
                                               method='closing', radius=2.0, show_points=True, fill_alpha=0.15)
for cid, env in envelopes.items():
    rmin, rmax, cmin, cmax = env['bbox']
    print(f"cluster {cid:3d}: {env['nb_points']:4d} pts, envelope area {env['area']:4d}, fill {100 * env['fill_fraction']:3.0f}%, "
          f"rows {rmin}-{rmax}, cols {cmin}-{cmax}, extent {env['extent']:.1f} steps, stray pts {env['nb_outliers']}")

cluster   0: 2243 pts, envelope area 2313, fill  97%, rows 0-47, cols 0-50, extent 67.9 steps, stray pts 0
cluster   1:  758 pts, envelope area  775, fill  98%, rows 14-44, cols 0-37, extent 39.9 steps, stray pts 0
cluster   2:  764 pts, envelope area  772, fill  99%, rows 20-50, cols 20-50, extent 37.6 steps, stray pts 0
cluster   3:  550 pts, envelope area  561, fill  98%, rows 15-36, cols 0-35, extent 36.1 steps, stray pts 0
cluster   4:  407 pts, envelope area  487, fill  84%, rows 0-31, cols 21-50, extent 36.1 steps, stray pts 0
cluster   5:  433 pts, envelope area  440, fill  98%, rows 4-25, cols 0-25, extent 28.2 steps, stray pts 0
cluster   6:  428 pts, envelope area  529, fill  81%, rows 28-50, cols 0-28, extent 33.8 steps, stray pts 0
cluster   7:  383 pts, envelope area  402, fill  95%, rows 33-50, cols 0-31, extent 32.3 steps, stray pts 0
cluster   8:  379 pts, envelope area  386, fill  98%, rows 31-47, cols 25-50, extent 28.0 steps, stray pts 0
cluster   9:  351 pts, envelope area  375, fill  94%, rows 10-27, cols 7-35, extent 28.6 steps, stray pts 0

fig.clear('all')
# extreme limits (convex hulls) on top of the KAM map
fig, ax, hulls = OC.plot_cluster_envelopes(result, analyzer, min_size=100, max_clusters=15, method='hull',
                                           show_points=False, fill_alpha=0.05, background=kam)
plt.show()

misorientation between neighbouring clusters

Pairs of clusters whose envelopes touch (contact) or overlap (overlap, grains superimposed along the beam or interpenetrating), misorientation between mean orientations and rotation axis (crystal frame of A). For cubic crystals, Σ3 twins (60° about <111>, Brandon criterion 8.66°) are identified.

pairs = OC.cluster_pairs_table(result, analyzer, cluster_ids=list(envelopes), envelopes=envelopes,
                               low_angle=5.0, high_angle=15.0)

   A    B misori(deg)       axis (crystal A)         type  dev.Σ3  overlap  contact
   0    4        2.83 [ 0.218  0.932 -0.290]    low-angle   57.67      486        1
   6    9        8.01 [-0.310 -0.412  0.857]       medium   52.79        0        3
   1    9       21.47 [ 0.362 -0.596  0.717]   high-angle   39.59      203       47
   1    6       25.24 [ 0.456 -0.754  0.473]   high-angle   35.93      295       43
   2    8       25.55 [ 0.276 -0.959  0.067]   high-angle   43.84      377        8
   3    7       26.21 [ 0.651 -0.307 -0.694]   high-angle   35.80        9       15
   1    3       26.29 [ 0.681  0.136 -0.719]   high-angle   38.45      446       32
   5    9       28.22 [-0.955 -0.030  0.294]   high-angle   43.16      210       33
   2    3       29.27 [ 0.660  0.168 -0.732]   high-angle   35.74       72       32
   6    7       32.65 [ 0.478  0.867 -0.139]   high-angle   35.82      329       35
   1    4       35.31 [-0.519  0.729  0.447]   high-angle   26.40       75       18
   0    1       37.53 [ 0.696  0.555 -0.455]   high-angle   23.82      763       10
   3    6       38.63 [-0.092 -0.873 -0.479]   high-angle   34.46      106       30
   4    8       39.17 [-0.347  0.550 -0.760]   high-angle   25.02        1        8
   3    8       39.21 [-0.694  0.193  0.694]   high-angle   28.63        0        3
   3    9       39.25 [-0.216 -0.923 -0.318]   high-angle   33.73      250       30
   1    7       39.28 [ 0.806 -0.470 -0.359]   high-angle   26.04      134       41
   0    8       41.08 [-0.330  0.593 -0.734]   high-angle   23.60      370       16
   1    8       41.36 [-0.127  0.861 -0.493]   high-angle   32.04       40       25
   1    5       43.07 [ 0.830 -0.399  0.391]   high-angle   24.54       92       43
   3    5       44.67 [ 0.453 -0.852 -0.262]   high-angle   26.72      202       31
   4    9       46.39 [ 0.896  0.443 -0.022]   high-angle   30.04      107       40
   1    2       47.37 [ 0.332  0.605 -0.724]   high-angle   19.42      188       45
   0    6       47.99 [-0.829  0.476  0.294]   high-angle   23.61      342       43
   0    9       48.17 [ 0.879  0.473 -0.054]   high-angle   27.79      375        0
   0    7       49.36 [-0.243 -0.624  0.743]   high-angle   22.43      196       48
   2    9       49.94 [-0.677  0.391 -0.624]   high-angle   15.20        3       10
   6    8       50.62 [-0.701  0.257 -0.666]   high-angle   21.03        7       13
   2    7       52.89 [ 0.562  0.444 -0.698]   high-angle   12.08      158       24
   2    6       53.77 [-0.645  0.514 -0.565]  Sigma3 twin    8.04       87       26
   3    4       54.69 [-0.743 -0.541  0.394]   high-angle   14.73       63       14
   0    2       56.51 [-0.587 -0.694 -0.416]   high-angle   11.63      645       52
   0    3       57.35 [ 0.380  0.754 -0.536]   high-angle   15.28      561        0
   2    4       58.40 [ 0.566  0.721  0.401]   high-angle   13.01       38       32
   4    5       58.41 [-0.604  0.479  0.638]  Sigma3 twin    6.90        1       22
   0    5       58.84 [-0.572  0.517  0.637]  Sigma3 twin    4.96      440        0
   7    8       60.20 [-0.686 -0.329 -0.649]   high-angle   16.10       44       20

boundary map

Boundaries between neighbouring points whose primary grains belong to different clusters, colored by misorientation class. Where several grains are superimposed, the primary grain (most indexed spots) may switch from point to point, which also produces boundary segments.

fig, ax, bnd = OC.plot_boundary_map(result, analyzer, low_angle=5.0, high_angle=15.0, background='clusters')