The architectures used in satellite image segmentation have passed through three generations in a decade, and each resolved a difficulty the previous one left open. Understanding that progression permits choosing without following fashion.
This article traces that evolution, sets out what each family addresses and what none resolves. It extends the article on satellite segmentation tools.
The problem these architectures solve
One tension structures the design of a satellite image segmentation model. It explains the shape of every architecture that follows. Recognising what an object is requires extended context. An isolated pixel says nothing; its meaning depends on its surroundings over a sometimes considerable distance. Precisely locating a limit requires the opposite, fine information preserved at the original resolution. Those two requirements oppose one another. Widening the context requires compressing information, which destroys localisation precision, and preserving that precision limits accessible context. Every architecture in this field is a response to that tension, and their differences lie in how each one arbitrates it.The first generation
Encoder-decoder architectures remain widely used. They dominated the first half of the decade. Their principle has three stages. An encoder progressively compresses the image into increasingly abstract and decreasingly resolved representations. A decoder then reconstructs a map at the original resolution. And skip connections link corresponding levels of the two branches, transmitting the fine information compression had lost. Those connections are the decisive innovation. They give the decoder simultaneous access to the context the encoder extracted and the detail compression would have destroyed. Three qualities explain their remarkable persistence. They train on modest volumes, which suits a field where corpora are scarce. They are robust and well understood. And they easily support an arbitrary band count, an important property in satellite imagery. One limit nonetheless characterises them. Their receptive field remains bounded, which makes them weak on classes defined by extended context, a difficulty the nomenclature article flagged for functional categories.The second generation
Attention-based architectures respond to that limit. They marked the second half of the decade. Their principle is to let each position in the image consider any other position directly, rather than propagating information step by step. Three clear advantages follow in satellite image segmentation. Extended context becomes accessible without compression, which addresses the tension set out above. Long-range relationships are captured, useful for classes whose definition depends on spatial organisation. And transfer between domains works better, these architectures lending themselves well to pre-training. Three serious drawbacks accompany them. The training data requirement is markedly higher, which sits in direct tension with the scarcity of geospatial corpora. Computation cost grows sharply with image size, which complicates handling large scenes. And interpretability decreases, which weighs in contexts where the method must be justified. One important practical consequence follows. These architectures suit a project holding a substantial corpus or able to start from a pre-trained model, and they are counterproductive on a small specialised corpus.The third generation
Geospatial foundation models are the current development. Their promise deserves examining precisely. Their principle is to pre-train a model on very large volumes of unlabelled imagery, having it solve artificial tasks built from the data itself, then to specialise it on a precise task with few labelled examples. Three contributions are genuinely real. The annotation volume needed for specialisation falls appreciably. Pre-training exploits the abundance of unlabelled imagery, a resource the Copernicus article documented. And specialisation is fast, which shortens a project’s cycle. Three limits are equally real and the AI article in the previous series documented them. The evaluation need remains entire, an annotated reference staying indispensable for measurement. The geographic bias of the pre-training transmits to the specialisation, these models being trained on corpora whose coverage is uneven. And specialisation still requires examples, in reduced number but of higher quality. A balanced reading is required on this point. These models shift the annotation need from quantity towards quality, which changes the nature of the work without removing its necessity.Choosing an architecture in practice
Four criteria permit deciding in satellite image segmentation. They differ from those the literature emphasises. Available corpus volume is the first. A modest corpus points to the first generation, a substantial one permits the second, and a specialised need on few examples points to the third. The extent of context needed is the second. A locally defined class is handled by a simple architecture, a class defined by its surroundings requires an extended receptive field. Computational constraint is the third, near real-time processing or very large areas excluding the heaviest architectures. And the explainability requirement is the fourth, a regulatory or contractual context potentially imposing a justifiable method, which the previous series documented. One observation runs across those four criteria. None concerns published performance on a benchmark, that figure being obtained under conditions that are not those of a particular project.What matters more than the architecture
This observation contradicts a widespread intuition. It deserves stating firmly. At equal corpus, the performance gap between two reasonable architectures is markedly smaller than the gap the corpus quality itself produces. Four factors weigh more than the architecture choice. The quality and consistency of the annotation, which the preceding articles detailed. The geographic and temporal representativeness of the corpus. The class definitions and the treatment of edge cases. And the correspondence between training conditions and deployment conditions. One practical consequence follows for a satellite image segmentation project. Effort spent on the architecture before those four factors are addressed is premature optimisation, and the reverse order produces a better result at lower cost. That observation carries commercial reach. It justifies a data provider steering a technical discussion towards the corpus rather than the model, and that redirection serves the client’s interest as much as its own.The difficulties specific to geospatial work
Five constraints distinguish training a satellite image segmentation model. They are poorly handled by standard practice. Chip-based processing, which the opening article set out, introduces border artefacts and a loss of extended context. The band count exceeds three, which precludes directly using weights pre-trained on natural imagery and requires adapting the first layer. Class imbalance is extreme on several useful classes, which the water and network articles detailed. The train and evaluation split must be geographic rather than random, two neighbouring chips sharing too much information to sit on either side. And data augmentation requires adaptation, some usual transformations being inappropriate, a flip altering solar orientation and a colour variation destroying the physical meaning of the bands. That last constraint is the most often neglected. An augmentation chain carried over from an ordinary vision project produces physically impossible examples, which degrades learning rather than enriching it.Handling extended context
This difficulty has no equivalent in ordinary vision. It is specific to satellite image segmentation. A natural image generally contains the whole object and its immediate context. A satellite scene contains thousands of objects and a context extending over kilometres, which chip-based processing necessarily truncates. Four responses are used and their costs differ appreciably. Enlarging the chips preserves more context and sharply increases computation cost, the relationship being quadratic. Degrading input resolution widens context at constant cost and sacrifices localisation precision, which returns to the original tension. Using two scales simultaneously, a degraded wide view for context and a fine view for detail, addresses the tension at the price of a more complex architecture. And adding separately computed contextual variables, position, elevation or neighbourhood statistics, supplies context without passing through the image. That last response is the cheapest and the most neglected. A simple variable indicating geographic position or elevation appreciably improves several classes in satellite image segmentation, and it requires no architectural modification.Transfer and its limits
This widespread practice has highly variable effectiveness. Starting from a pre-trained model rather than a random initialisation accelerates learning and reduces the data requirement. Three quite distinct situations occur. Pre-training on natural imagery brings a real gain on visible bands and nothing to infrared bands. Pre-training on satellite imagery brings more and it transmits the geographic bias of its corpus. And pre-training on the same geographic area is the most favourable case and it is rarely available. One important caveat nonetheless frames this practice. Transfer between geographies is limited, research having documented it, which means a model pre-trained on one region and specialised on another retains part of the initial bias. One direct consequence follows for a corpus. Building examples in the deployment geography matters more than increasing volume in another, an observation matching the central finding of the previous series.The temporal dimension
Part of satellite image segmentation concerns the series rather than the image. The vegetation article showed it. Handling a time series requires a different architecture, the distinguishing information residing in the evolution rather than in the appearance. Three approaches are distinguishable. Stacking dates as additional bands, simple to implement and assuming a fixed and regular number of dates. Recurrent architectures, which handle a sequence of variable length and manage gaps naturally. And temporal attention architectures, which weight dates by relevance and suit irregular series well. One difficulty dominates all three. Optical series carry irregular gaps from cloud, which the vegetation article detailed, and the handling of those gaps must be identical between training and production or the model is presented with an unfamiliar structure. One practical consequence follows for a temporal corpus. The annotation concerns a whole series rather than a date, which changes the unit of work and requires documenting the dates available for each example.Evaluating a satellite image segmentation architecture
Four precautions distinguish a rigorous comparison from a misleading one. Comparing on the same corpus, a difference in data producing a gap larger than the architectures’. Evaluating on areas geographically separated from training, failing which the measurement reflects memorisation. Reporting boundary metrics alongside global ones, the corresponding article having shown that two models with identical scores can produce very different contours. And measuring computation cost alongside performance, a marginal gain obtained at multiplied cost not necessarily being desirable. One important observation completes those four precautions. Performance published on a public benchmark tells you about the model’s behaviour under those conditions and not about its behaviour in a particular project, a distinction the previous series documented as the gap between announced and real-world performance.What the architecture choice changes for a corpus
This inverse reading directly concerns a data provider. The architecture adopted modifies the requirements bearing on the corpus, which makes this question a matter of data as much as of modelling. Four effects are distinguishable. The volume needed varies strongly, a data-hungry architecture requiring several times the examples of a frugal one for comparable performance. The distribution of examples matters more on extended-context architectures, which learn spatial relationships and not only local appearances. The size of annotated chips must match the intended receptive field, a corpus built in small vignettes not permitting an architecture exploiting wide context to be trained. And boundary quality takes on greater importance on resolution-preserving architectures, which learn precisely the contours shown to them, noise included. One important practical consequence follows. Establishing the intended architecture before building a corpus avoids producing unsuitable data, and that question belongs to joint scoping between the data provider and the modelling team rather than to either separately.The reproducibility question
One requirement is gaining importance and it concerns the corpus as much as the model. Reproducing a result requires recovering four elements, and the absence of any one makes the exercise impossible. The exact corpus version used, a corpus corrected mid-project producing different results with no model change having occurred. The split between training, validation and evaluation, whose random reconstitution does not restore the original partition. The preparation parameters, tiling, normalisation and augmentation, whose effect is as decisive as the architecture’s. And the versions of the libraries used, an update sometimes altering a default behaviour. One practical consequence follows for a satellite image segmentation project. Those four elements are recorded during the work and cannot be reconstituted afterwards, a principle every cluster in this series established about documentation.Approaching an architecture discussion
A data provider can participate usefully. Four questions permit it, on a satellite image segmentation subject where it is not the principal actor. What volume of annotated examples is planned, a question determining which architectures are realistic and one belonging directly to the provider. What chip size will be used, information necessary to produce a compatible corpus and rarely communicated upfront. Will the train and evaluation split be geographic, a question conditioning how the corpus must be organised from the outset. And what augmentation chain will be applied, information permitting inappropriate transformations to be flagged before they degrade learning. Those four questions share a common property. They concern the interface between the data and the model, a domain where a provider holds a competence the modelling team does not always have, and where an error costs the entire corpus.What no architecture solves
Architectural progress is regularly presented as a universal answer. No architecture compensates for a poorly defined nomenclature, a model learning to reproduce the inconsistency shown to it. None compensates for a noisy reference, the inter-annotator disagreement bound applying to all. None fills in absent information, a use not observable in the image not becoming predictable through a better architecture, as the vegetation article showed. And none transfers to a geography unrepresented in its training, a limit research documented and architectural progress has not reduced. Those four limits share a common property. All belong to the data and its definition, which explains why a project’s most profitable effort concerns the corpus rather than the model.What this survey teaches
This observation explains a singularity of satellite image segmentation. The architectural progress of the last decade has been considerable and its effect on real projects has been modest, a gap demanding an explanation. Three reasons produce it. The published gains are obtained on benchmark corpora whose bias the AI article in the previous series documented, which makes them poorly representative of project conditions. The best performing architectures require data volumes few geospatial projects hold, which makes them inapplicable where the need is greatest. And a project’s dominant difficulties, nomenclature, boundary conventions and representativeness, are external to the architecture and have therefore not benefited from that progress. One consequence follows for a provider. The value it brings sits precisely where technical progress has not reached, which makes its position more stable than the rhythm of publications suggests.When a simpler architecture is the right answer
One satellite image segmentation recommendation deserves stating plainly, since the direction of fashion runs against it. Four situations favour the first generation over anything newer, and they cover a large share of real projects. A corpus below a few thousand annotated examples, where a data-hungry architecture underperforms a frugal one regardless of its theoretical advantage. A locally defined class, where extended context adds nothing and the cost of accessing it is pure overhead. A deployment under computational constraint, where inference cost matters as much as accuracy, which the near real-time article set out. And a context requiring explanation, where a well understood architecture is defensible in a way a recent one is not. One observation completes those four. Choosing an older architecture for good reasons is a defensible technical decision; choosing a newer one because it is newer is not, and the second happens more often than the first. The practical test is simple. If the reason for preferring an architecture cannot be stated in terms of the project’s constraints, the preference is aesthetic rather than technical, and the budget is better spent on the corpus.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Choosing an architecture on published performance rather than on project constraints.
- Using a data-hungry architecture on a modest corpus.
- Carrying over an augmentation chain designed for natural imagery.
- Splitting train and evaluation randomly rather than geographically.
- Using pre-trained weights without adapting the first layer to the available bands.
- Neglecting class imbalance in the loss function.
- Optimising the architecture before addressing corpus quality.
- Comparing two architectures on different corpora.
- Expecting a foundation model to remove the evaluation need.
- Assuming a pre-trained model transfers without bias to a new geography.
- Building a corpus in small vignettes for an extended-context architecture.
- Handling a time series with an architecture designed for a single image.
