When structural disagreement did not transfer

Abstract

Deterministic structural descriptors may complement existing remote-sensing models through prediction, review prioritization or measurement audit. We evaluated these roles using co-located RGB, hyperspectral and lidar products in an archived NEON benchmark. Exploration progressed from one plot to five discovery plots, followed by all 108 eligible plot-years at 11 sites. The archive contained 1,597 eligible stems, of which 628 full-sun/open-grown trees formed the primary analysis. An exploratory review rule combining detector confidence with local centroid-disagreement variation reduced equal-site retained-case error from 1.268 to 0.953 m at a nominal 40% review budget in discovery. Applied unchanged to 112 exposed trees at seven fresh sites, with a crown-size predictor fitted only on discovery plots, it yielded 2.015 m versus 2.010 m for confidence alone. In leave-one-site-out prediction, adding the Structural Topology Kernel (STK) reduced pooled mean absolute error from 1.924 to 1.869 m, but effects varied by site. A fuller structural bundle added only 1.1% pooled improvement beyond ordinary texture. Structural additions worsened two operational audit classifiers. These results delimit the usefulness of a specific deterministic add-on; they do not establish general operational benefit or reject deterministic measurement as a research approach.

Keywords: NEON; tree crowns; deterministic descriptors; multisensor disagreement; geographic transfer; review prioritization; negative results.

1. Question and context

The objective was partnership: determine whether fixed structural measurements add information to a practical predictor or its review process. Deterministic extraction can support traceability, but repeatability is distinct from accuracy, input robustness and useful error ranking. Fixed neural inference may also be repeatable; the relevant comparison is added value at the same evaluation conditions.

The NEON benchmark combines imagery and several reference-data types for crown evaluation [1]. Geographic transfer is already a recognized challenge in tree detection [2], DeepForest provides an established RGB detector [3], and crown annotation imprecision has motivated dedicated evaluation metrics [4]. Our contribution is a specific, documented add-on experiment and its discovery-to-transfer outcome. We do not claim a new benchmark, a comprehensive detector comparison or established novelty of every descriptor.

Three questions were separated: does structure improve field-measured crown-size prediction; does it improve selection of cases for review while leaving predictions unchanged; and does it improve discrimination of operational matching or height-inconsistency labels?

2. Data and methods

2.1 Cohort, target and chronology

Plot products were pinned to NEONTreeEvaluation commit 96b29d566ca7ea9f604de3e1235c503ab101cb8a. Eligibility required live status, finite coordinates and height, height >3 m, stem diameter >15 cm, two positive crown diameters, and overlapping plot-year sensor products. Duplicate individual-year records with conflicting physical measurements were removed; identical repeated records were collapsed. Year came from the field event identifier. The response was D = √(d_max × d_90°), in metres. It is a scalar field crown-size measurement, not a reference crown mask.

The initial field probe used 58 stems at NIWO. Five discovery plots subsequently supplied 163 eligible stems, including 98 exposed trees from four sites. The broad run covered 108 plot-years and 1,597 stems at 11 sites. Its primary population comprised 628 trees recorded as “Full sun” or “Open grown,” retaining unique, ambiguous and unmatched detections. Seven sites contributed 112 primary cases not used in discovery; another 418 primary cases came from additional plots at the four discovery sites. Primary training in all regression folds used exposed trees only.

All 415 broad-run sensor files, totaling 228,554,822 bytes, were checked against pinned Git blob hashes. A field-to-raster UTM mismatch at BLAN initially excluded 71 stems. Reprojecting eligible stems and the occupancy inventory from their recorded UTM zones to the raster CRS recovered all 71, including 18 primary cases. This correction followed an initial scoring pass and is explicitly documented. Final analysis retained all eligible stems under the patch-coverage rules; no outcome-based exclusions were introduced.

2.2 Sensor fields and deterministic measurements

RGB imagery had 0.1 m pixels; hyperspectral and aligned height fields used a 1 m grid. RGB was averaged onto that grid. Hyperspectral crops inherited the co-located RGB CRS, a provisional metadata assumption. Normalized excess green was (2G−R−B)/(R+G+B), and NDVI used bands 58 and 94, approximately 668 and 849 nm. Supplied canopy-height models were averaged to the common grid. Where absent, canopy height was estimated from maximum nonnoise lidar height above a ground surface interpolated from classified ground points, with nearest-ground fallback.

Nearest-value filling addressed missing grid cells, and fields were clipped to nonnegative values before structural calculations. Eligibility required a complete 7 × 7 m sensor patch, a complete native 70 × 70 pixel RGB patch, and at least 90% common valid cells before filling. Filling and clipping are part of the instrument definition, not learned corrections.

Here A_k is a nonnegative excess-green, NDVI or height field and coordinates are in metres. The implementation assigns centroid (0,0) when total mass is ≤10⁻¹²; that convention can affect low-signal windows. Ω was evaluated on every overlapping 3 × 3 m and 5 × 5 m subwindow of a 7 × 7 m patch: 25 and nine windows, respectively. The two modeled disagreement features were population standard deviations of these Ω values. “Variation” therefore means standard deviation, not variance, and overlapping windows do not create independent observations.

The Structural Topology Kernel (STK), implemented in the archived vtl_materials_kernel.py, was applied to native grayscale patches using luminance weights (0.2126, 0.7152, 0.0722). Its fixed implementation normalizes and measures geometry, radial response, texture, blobs and related structure. All 50 outputs were retained. Four coordinates were excluded from modeling: two trajectory metrics affected by an identified np.diff(pts, 0) implementation issue, and two graph metrics with insufficient support at this scale. The remaining 46 coordinates were reduced to five training-fitted principal components. No source-library repair was silently substituted.

Comparators included six ordinary grayscale texture measures (mean, standard deviation, gradient-magnitude mean and standard deviation, and horizontal autocorrelation at lags one and five pixels), plus 15 morphology features: standard deviation, dispersion and anisotropy for each sensor, and pairwise centroid distances and absolute dispersion differences. Dispersion is the square root of the weighted covariance trace; anisotropy is the covariance eigenvalue difference divided by their sum, with denominator floor 10⁻⁹.

2.3 Predictor, detector and matching

The local predictor used six features: height at the stem, local height mean and standard deviation, mean excess green, mean NDVI, and nearest eligible-stem distance. It is a field-linked research baseline requiring known stem positions and an eligible-stem inventory. Median imputation, standardization and PCA were fitted within training folds. Ridge regression used log D with fixed penalty 10; predictions were exp(clamp(predicted log D, −4, 5)). No penalty or feature-subset search was conducted in the broad run.

A frozen weecology/deepforest-tree checkpoint at revision cc21436bc5d572dde8ff5f93c1e71a32f563cace was loaded strictly through a torchvision RetinaNet adapter. Score and nonmaximum-suppression thresholds were 0.5 and 0.05. No fine-tuning was performed. A unique operational match required exactly one box containing the stem and exactly one mapped live stem >3 m within that box. Occupancy used the broader archived inventory; unmatched trees were not force-assigned. Confidence risk was 1−score of the highest-confidence containing box; unmatched stems received confidence zero. This detection score is not calibrated uncertainty for the crown-size regressor.

2.4 Evaluation and review rule

Broad prediction comparisons held out one entire site at a time. Both pooled MAE (each tree equally weighted) and mean site MAE (each site equally weighted) were reported. The review transfer test instead fitted the local predictor only on the five discovery plots and 98 exposed stems, then froze it. This makes the fresh-site evaluation distinct from the all-site regression experiment.

High R indicated review priority. Ranks used average ties and the unlabeled site batch; neither labels nor error values set the ranks. The rule used equal weights without a weight search. At each nominal budget b∈{0.2,0.4}, ceil((1−b)n) lowest-risk cases were retained. Boundary ties were evaluated by exact expected error under uniform selection within the tied group. Comparators used identical per-site retention counts. No reviewer corrections, time savings or reductions in total delivered error were simulated.

The confidence-plus-Ω rule was developed after inspecting the discovery outcomes and then frozen before broad scoring. It was not a preregistered confirmatory finding in discovery. The broad plan required improved fresh-site equal-site and pooled retained MAE at 40%, with a majority of sites improving, to retain the general review lead. Whole-site bootstrap intervals used 5,000 resamples of paired site differences; they are descriptive, involve few sites and carry no multiplicity adjustment. A ≥10-case/site sensitivity was added after initial scoring and does not replace primary results.

Two exploratory leave-site-out logistic classifiers tested nonunique operational matching and absolute field–lidar height discrepancy >4 m. The baseline used local sensor features and ordinary texture; augmentation added Ω, morphology and five STK components. Transforms were training-fitted; logistic regression used C=1 and balanced class weights. Matching status and field height supplied labels, not predictor inputs. Equal-site AUC included only sites containing both classes.

3. Results

3.1 The review advantage did not transfer

At a 40% nominal review budget, discovery retained-case MAE fell from 1.268 to 0.953 m when Ω was added to confidence. The same rule on seven fresh sites yielded 2.015 versus 2.010 m for confidence alone. Forty-one of 112 cases were sent for review. Three sites improved, three worsened and one tied. The equal-site gain was −0.0048 m, with bootstrap interval [−0.137, 0.099] m.

The tiny pooled improvement at 40% did not satisfy the declared combined criteria. At 20%, equal-site MAE improved from 2.044 to 1.998 m, but only two sites improved and its gain interval also crossed zero [−0.068, 0.173] m. On additional plots at discovery sites, 40%-budget equal-site retained MAE was 1.915 m for confidence and 1.933 m for confidence + Ω. In the all-site leave-site-out evaluation it was 1.893 versus 1.948 m. These results provide no consistent basis for a general review advantage.

Absolute discovery and transfer errors are not directly comparable: their populations differ, and discovery used site-held-out predictors whereas transfer used a predictor fitted to all discovery plots. The relevant evidence is the paired policy comparison within each evaluation.

3.2 Prediction gains were modest and heterogeneous

STK reduced pooled MAE by 2.8% and equal-site MAE by 3.4%. The equal-site gain of 0.067 m had interval [−0.149, 0.325] m; six sites improved. BLAN and SERC improved by 1.075 and 0.509 m, while MLBS, UKFS, GUAN and KONZ worsened. Excluding BLAN only from the summary of the same saved predictions reversed the mean gain to approximately −0.034 m; no models were refitted for that influence check.

Ordinary texture achieved 1.834 m pooled MAE. The combined texture/structural bundle reached 1.813 m, a further 1.1% pooled improvement, with gains at five of eleven sites relative to texture. Its equal-site gain interval [−0.037, 0.259] m crossed zero. The bundle's 5.7% improvement over the bare local baseline should not be attributed entirely to structure because ordinary texture already captures much of it.

3.3 Operational audit classifiers did not improve

Structural augmentation lowered equal-site AUC for both tasks. These outcomes concern operational proxies: nonunique containment is not independently adjudicated crown error, and field–lidar height differences may reflect real canopy geometry as well as measurement issues.

3.4 Earlier diagnostics constrain interpretation

The initial NIWO lidar segmentation placed 57 of 58 stems in shared segments, limiting a single-crown interpretation of patches. In the five-plot experiment, a 30-tree crown-centered structural model appeared to improve pooled MAE from 1.015 to 0.777 m against a detector-augmented baseline, but the simpler local baseline already achieved 0.749 m. Moving centers by 1 m raised the structural model's MAE to 0.936–0.970 m. Those secondary findings do not establish a stable crown-specific effect.

A separate LENO image-label check gave 81.1% precision and 32.5% recall at IoU≥0.4 for the frozen detector, indicating conservative detections on that scene. It is a diagnostic, not a full-archive detection benchmark. Synthetic controls recovered a planted centroid translation and expected symmetric-blur behavior; cropping changed the disagreement read. Repeat STK extraction on one patch per discovery site matched exactly in the tested runtime. These checks support implementation traceability while exposing scale and support limitations.

4. Discussion

The central contribution is a negative transfer result for a specific review rule, accompanied by a smaller and uneven prediction-feature result. The study illustrates why deterministic measurements should be evaluated against the partner's intended task. A reproducible structural discrepancy can exist without identifying a consequential prediction error.

One plausible explanation is that optical greenness and canopy height measure different properties. Their centroids can diverge for meaningful canopy reasons rather than because an input or prediction is wrong. Stem-centered windows, shadow, neighboring crowns and spatial resolution may further alter the relationship. The present experiment does not distinguish these mechanisms, and it does not estimate independent sensor-error variances.

The prediction results suggest some complementary information in STK, particularly at certain sites. They do not establish broad robustness or practical utility. The site influence, uncertainty intervals and texture comparison argue against presenting a single pooled percentage as a general forestry advance. Likewise, failure of the fixed Ω rule does not imply that every disagreement measure or review policy must fail.

Limitations

The cohort is a selected subset of one archived benchmark, not a random sample of forests. Stems are clustered within 108 plot-years and only 11 sites; GUAN and KONZ contribute four and two primary cases. The frozen detector may have prior exposure to NEON sites, so downstream site holdout is not proof of unseen detector-training geography. Its single-scene low recall and the absence of extensive comparator tuning limit claims about strong operational baselines.

Field crown axes and stem locations do not establish exact image crown boundaries or perfect temporal alignment. HSI CRS inheritance, nearest filling, positive clipping and fixed spatial windows are substantive assumptions. The BLAN coordinate correction followed initial scoring. Several hypotheses and secondary controls followed earlier outcomes; the complete sequence is retained in the labbook. Bootstrap intervals are descriptive, not a multiple-testing-corrected confirmation or proof of no effect.

Practical implication

Retire the tested confidence-plus-Ω rule as a general review aid for this workflow. Preserve STK and the combined descriptors as modest candidate features with their site-specific outcomes attached. Further parameter searches on the same archive would not resolve transfer or label independence. A subsequent domain study should define what each sensor measures and use an independently measured error endpoint appropriate to the proposed partnership role.

5. Conclusion

A promising discovery-stage review signal did not provide consistent added value on new sites. Deterministic structural features produced modest, heterogeneous crown-size prediction improvements, and the larger bundle added little beyond ordinary texture. The contribution is a bounded empirical case study and reusable analysis record. Operational usefulness, novelty of the specific approach and wider ecological implications remain unestablished.

6. Research record and reproducibility

This note was written from saved outputs; no new model fitting was performed during drafting. The accompanying consolidated research record preserves six chronological entries, the research charter, protocols and amendments, all analysis outputs within its declared scope, scripts, feature arrays, per-plot records, manifests and source hashes. The supplement includes a runtime inventory and the STK source and naming documentation. Download manifests identify the raw public products and frozen checkpoint.

Raw sensor rasters and point clouds, model weights, the virtual environment, and external library installations are not embedded. Some scripts retain original local paths and need adaptation for another machine. An independent clean-environment reproduction and a fuller novelty review have not been completed. This document is a technical research note draft, not a claim of peer review or publication readiness.

Numerical checks reproduced stored prediction aggregates, review tie expectations and source hashes; geometry checks verified the final complete patches and BLAN reprojection. Full formal definitions of the archived STK coordinates are given by its preserved source implementation. The term STK means Structural Topology Kernel; an earlier consolidated brief incorrectly expanded it as “Structural Texture Kernel,” now corrected without changing calculations.

Appendix: cohort and feature specification

The discovery-site pool includes the original five plots and later plots from those sites. Fresh-site status refers to downstream analysis only.

46 retained STK coordinates: delta_x, delta_y, r_v, mu, rho_r, x_p, theta, d_s, sdi, g2_void, g2_nvoids, g2_AR, g2_cut, g3_kvar, g3_infl, g4_oent, g4_slope, sci, rc_drc, rc_dr, tex_contrast_fine, tex_entropy_fine, tex_homogeneity_fine, tex_energy_fine, tex_contrast_coarse, tex_entropy_coarse, tex_homogeneity_coarse, tex_energy_coarse, blob_count, blob_density, blob_size_mean, blob_size_cv, n_islands, island_size_cv, island_spread, ds_low, ds_mid, ds_high, edge_density, nils, ler, gap_rv, gap_theta, tonal_shadow, tonal_mid, tonal_high.

Excluded: g1_pathlen, g1_curvature, resistance_mean, drain_confidence. Raw outputs remain in the record. STK feature extraction is deterministic; the PCA, regression and classifier components are fitted.

References

1. Weinstein BG et al. (2021). A benchmark dataset for canopy crown detection and delineation in co-registered airborne RGB, LiDAR and hyperspectral imagery from the National Ecological Observatory Network. PLoS Computational Biology 17(7): e1009180. Article.
2. Weinstein BG et al. (2020). Cross-site learning in deep learning RGB tree crown detection. Ecological Informatics 56: 101061. Author manuscript.
3. Weinstein BG et al. (2020). DeepForest: A Python package for RGB deep learning tree crown delineation. Methods in Ecology and Evolution. Article.
4. Stewart D et al. (2021). RandCrowns: A Quantitative Metric for Imprecisely Labeled Tree Crown Delineation. arXiv:2105.02186.
5. Local archived instrument: Structural Topology Kernel — code & data release, README_STK.md and vtl_materials_kernel.py, ZTOYBOX/Material. Source hashes and copies are included in the consolidated analysis supplement. This is code provenance, not a claim of an independently validated forestry instrument.