Buildings present difficulties exactly opposite to those of water in satellite image segmentation. Where a water surface poses a problem of definition and temporal variability, a building has sharp limits, a stable existence and a definition that invites little argument.Its difficulty lies elsewhere: in geometry, in density and in the gap between what is seen and what is to be measured. This article sets out those difficulties and the conventions a project must fix. It extends the article on segmenting water surfaces.
What makes buildings different in satellite image segmentation
Five properties distinguish this class and they shape the whole method.Building limits are sharp. A wall is an abrupt transition, which removes the boundary ambiguity dominating most other classes.Built objects are countable. A building is an individual entity, which makes instance segmentation relevant where land cover is content with semantic segmentation.Temporal stability is high. A building rarely changes, which keeps an annotation valid for a long time and shifts interest towards change detection.Building geometry is regular. Buildings mostly present right angles and simple shapes, a property exploitable for regularisation.And the third dimension intervenes. A building has a height, which a two-dimensional image shows only indirectly and which produces the difficulties set out below.The central difficulty of building segmentation
This geometric discrepancy dominates the domain and it is poorly understood by teams coming from ordinary computer vision.A satellite image is taken from an angle that is never exactly vertical. A building therefore appears leaning, its roof offset from its ground footprint, increasingly so as the building is taller and the viewing angle departs from vertical.Three quite distinct geometric objects result, and confusing them produces most of the disagreement in this domain.The ground footprint denotes the surface occupied by the building at ground level. It is what cadastral and planning uses require.The roof outline denotes what is visible on the image. It is what an annotator spontaneously delineates.And the apparent envelope includes the visible facade, meaning the union of the previous two.One practical consequence follows immediately. The convention must be fixed explicitly, and the ground footprint is not directly observable on an oblique image, which requires either a geometric correction or a convention that accepts the discrepancy.The other geometric difficulties
Four difficulties complete the previous one and they compound in dense settings.Cast shadows darken neighbouring buildings and can entirely mask low constructions at the foot of tall blocks.Occlusions hide buildings behind others, a frequent situation in dense fabric viewed at a pronounced angle.Party walls remove the visible separation between adjacent buildings, which makes instance segmentation difficult where semantic segmentation poses no problem at all.And courtyards and patios raise the question of their inclusion in the footprint, a decision belonging to convention rather than to observation.Those four difficulties share a common property. They all intensify with urban density and with departure from vertical, which makes model performance strongly dependent on urban context and acquisition angle.The resolution question
This constraint settles the feasibility of a satellite image segmentation project.It is stricter here than on other classes. The general rule set out in the opening article applies severely: delineating a building means distinguishing its limits, which requires several pixels across its smallest dimension.Three resolution bands are distinguishable in practice. At ten metres, only large built ensembles are discernible, and the product belongs to land cover rather than to buildings. At a few metres, medium-sized individual buildings are detectable but their outline remains approximate. And below one metre, individual delineation becomes precise.An economic consequence follows. A project mapping individual buildings presupposes commercial or aerial imagery, which shifts the cost question towards acquisition rather than annotation, unlike most other classes.Semantic or instances
This choice structures the entire project.It determines both the cost and the possible use of the product.Semantic satellite image segmentation produces a map of built-up area without distinguishing constructions. It suffices to measure a land take rate, to map an urban footprint or to feed an aggregate risk model.Instance segmentation produces one object per building. It is necessary for counting, for attaching individual characteristics and for any exploitation in an object-oriented geographic information system.Three considerations concretely orient that choice. Annotation cost, the second being markedly higher since it requires separating adjoining buildings. The density of the fabric, separation becoming very difficult in a historic centre. And the use, the decisive question being whether the client will count objects or measure surfaces.One recommendation follows. Asking that question at scoping avoids considerable extra cost, a client frequently asking for instances out of habit where a surface rate would answer their need.Contour regularisation
This operation distinguishes a raw product from a deliverable one.Pixel-based satellite image segmentation produces irregular contours following pixel boundaries, whereas a building presents straight edges and mostly right angles.Regularisation means straightening those contours by exploiting that regularity, which improves both the appearance and the geometric precision of the result.Three precautions frame that operation. Overly aggressive regularisation removes real detail, a legitimate extension or setback disappearing in favour of a rectangle. Non-orthogonal buildings, curved or complex, are degraded by regularisation assuming right angles. And the parameters must be documented, since they introduce a deliberate geometric discrepancy between the result and the observation.One consequence for evaluation deserves stating. Regularisation modifies overlap metrics, sometimes downwards while the product is visually better, which requires distinguishing geometric quality from pixel conformity.Change detection in satellite image segmentation
This is the economically strongest use of this class.The stability of buildings shifts interest towards evolution.Three types of change are distinguishable, with very different difficulties.New construction is the most detectable, a building appearing where there was none.Demolition is equally clear, with the difficulty that the bare ground remaining resembles a construction site.And extension or modification is the hardest, a partial change to an existing building requiring fine geometric comparison and being highly sensitive to geolocation discrepancies between the two dates.That last sensitivity is absolutely decisive. A geolocation offset between two acquisitions produces an apparent change on every building, which drowns real changes in systematic noise and makes prior registration indispensable.One consequence for annotation follows. A change detection corpus must contain no-change cases with geometric offset, failing which the model learns to flag any offset as a change.Available reference sources
This particularity changes the economics of a satellite image segmentation project.Buildings are the class for which the most pre-existing references exist, which is rare in this field.Three families are available to a project. Cadastral and topographic references produced by administrations, generally precise and sometimes freely accessible. Collaborative databases, whose coverage is very uneven geographically and whose quality varies. And datasets from research projects, often limited to a few cities.Four precautions necessarily frame their use as reference. The date, a reference potentially lagging the imagery by years. The geometric convention, a cadastre describing a ground footprint where the image shows a roof. Exhaustiveness, some constructions escaping official references. And coverage, availability being very uneven between countries.One practical consequence follows. Using an existing reference as ground truth is legitimate and requires measuring its discrepancy against the imagery on a sample, failing which the evaluation attributes to the model discrepancies that come from the reference.Uses and their requirements
Five uses employ this class in satellite image segmentation, with markedly different geometric requirements.Land take monitoring demands wide coverage and temporal consistency, the trend mattering more than individual precision.Risk exposure assessment demands individual footprints with attributes, height and use notably, which the insurance article in the previous series described.Cadastral updating demands the highest geometric precision and a strict ground footprint convention, the roof offset not being acceptable there.Population estimation demands counting and volume estimation, which presupposes instances and a height.And post-disaster crisis mapping demands a state assessment rather than a delineation, a different exercise calling for a documented damage scale.The height question
This dimension escapes two-dimensional satellite image segmentation.It is nonetheless demanded by several uses.Three approaches allow it to be estimated, with very different reliabilities.Cast shadow supplies an estimate from its length, the acquisition time and the solar position. The method is simple and it fails in dense fabric, where shadows overlap, and under cloud.Parallax between two views supplies a stereoscopic estimate, more reliable and requiring suitable acquisition.And an external source, a digital surface model or LiDAR data, supplies the most reliable measurement, which explains the complementarity between imagery and LiDAR the next cluster of this series details.One practical consequence follows. A client asking for heights from imagery alone must be informed of the associated uncertainty, and the cheapest solution is often to cross-reference an existing elevation source rather than to estimate.The attributes attached to buildings
A satellite image segmentation client rarely asks for geometry alone.Four families of attribute are requested and their difficulty rises markedly.Geometric attributes, area, perimeter and compactness, are computed directly from the contour and require no additional work.Roof attributes, apparent material, condition and presence of equipment, are read from the image and require a nomenclature of their own with its edge cases.Structural attributes, height and storey count, require a complementary source as the previous section set out.And use attributes, residential, commercial or industrial, are not read from the image and belong to the distinction the nomenclature article drew between cover and use.That fourth family requires a commercial precaution. A use is partly inferred from context, shape and surroundings, which produces a plausible prediction rather than an observation, and that distinction should be set out to the client rather than masked by a flattering average performance.Evaluating a building product
The usual satellite image segmentation metrics report poorly on what matters here.Three levels of evaluation coexist and a project gains from reporting all three.Pixel evaluation measures overlap between predicted surfaces and reference. It is dominated by large buildings and it masks failures on small ones.Object evaluation measures how many buildings are correctly detected, applying an overlap threshold to decide that a predicted object matches a reference object. It reveals the merges and fragmentations a pixel measure does not see.And geometric evaluation measures contour quality independently of detection, through a distance between predicted and reference contours.Four distinct failure modes deserve tracking separately. Merging, several buildings detected as one, frequent with party walls. Fragmentation, one building detected in several pieces, frequent on complex roofs. Systematic offset, which signals a georeferencing problem rather than a model one. And omission of small buildings, invisible in a surface metric.That decomposition is more informative than a single figure, and it directs correction where a global score indicates nothing.The geographic variability of buildings
This limit is stronger on this class than on most others.A building does not look the same across regions, materials and construction practices, which strongly limits a model’s transferability.Five factors of variation deserve distinguishing. Roof materials, whose spectral signature differs considerably between tile, sheet metal, concrete and vegetation. Urban morphology, a regular suburban fabric and dense informal settlement having no common structure. Characteristic size, which varies substantially between contexts. Density, which conditions party walls and occlusions. And the immediate environment, a building on bare ground standing out clearly where one under tree cover is partly masked.One practical consequence follows, documented by research in this field. A model trained on European cities performs poorly on informal settlement, and the gap is not marginal but can render the product unusable.That limit is precisely the opportunity the previous series identified. The regions least covered by existing corpora are those where the mapping need is greatest, and a provider able to build a reference there holds what is missing.The cost of annotating buildings
This class has a particular cost structure in satellite image segmentation.Three characteristics distinguish it from other classes.Object density is the dominant factor. A dense urban scene contains thousands of buildings where a rural scene contains a few dozen, which makes cost per square kilometre vary by a considerable ratio.The geometric precision required is the second factor. An approximate contour and a metre-accurate one do not take the same time, and the gap is sharper here than on classes with diffuse limits.And separating party-wall buildings is the third. It turns a delineation task into a decision task, the annotator having to determine where a limit invisible on the image runs.Two levers effectively reduce that annotation cost. Automatic pre-annotation works particularly well on this class, existing models producing correct contours on isolated and regular buildings, which concentrates human work on the difficult cases. And using an existing reference as a starting point, where one exists, turns production into verification and updating work.Those two levers make this class economically more accessible than its apparent complexity suggests, provided the conventions have been fixed.The conventions to fix
Eight decisions must be settled before production.Omitting them produces substantial variability.The object delineated, ground footprint or roof outline, the most decisive decision.The treatment of roof overhangs, projections and canopies.The inclusion of outbuildings, garages, sheds and conservatories, under an explicit size threshold.The separation of party-wall buildings, or its acknowledged abandonment.The treatment of courtyards and patios.The minimum area threshold below which a building is not represented.The treatment of buildings under construction, a site belonging to a convention about completion stage.And the treatment of non-building structures, greenhouses, agricultural sheds and technical installations, whose inclusion depends on the purpose.What this use case teaches
This observation completes the one drawn from the water case in satellite image segmentation.Water illustrated a difficulty of definition. Buildings illustrate a difficulty of geometry, and the two call for opposite responses.On water, effort goes into nomenclature: establishing what the product describes resolves most of the problem.On buildings, definition poses no difficulty at all and effort goes into geometric conventions: which object is delineated, where an invisible limit runs, what becomes of a roof overhang.That opposition has a practical consequence. A team experienced on one class is not experienced on the other, and a protocol designed for one and transposed to the other leaves open precisely the decisions that matter.One general recommendation follows. Identifying at scoping whether a class’s dominant difficulty is definitional or geometric directs methodological effort where it produces an effect, and that question takes a minute to ask.Where the two clusters meet
One connection deserves stating, since the height question above pointed to it.Buildings are the class where satellite image segmentation most obviously reaches its limit. Everything difficult about them, the leaning caused by viewing angle, the invisible ground footprint, the unmeasurable height, the occlusions in dense fabric, follows from projecting a three-dimensional object onto a two-dimensional image.Airborne LiDAR, which the next cluster of this series covers, does not have that problem. It measures three dimensions directly, which makes the ground footprint observable, the height measured rather than inferred, and occlusion a matter of scan geometry rather than of interpretation.Three consequences follow for a project. Where budget permits, combining imagery and elevation resolves several of the difficulties listed above at once rather than mitigating them separately. Where it does not, the imagery-only conventions must be explicit, since they encode an approximation the client should know about. And a provider competent in both modalities can propose the arrangement that actually fits the requirement rather than the one its tooling supports.That is the practical reason these two clusters sit next to each other, and buildings are the class where the complementarity is clearest.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Confusing ground footprint with roof outline.
- Neglecting the effect of viewing angle on tall buildings.
- Asking for instances where a semantic map would suffice.
- Targeting individual buildings at a resolution that does not distinguish them.
- Regularising too aggressively and removing real detail.
- Not documenting regularisation parameters.
- Omitting prior registration in change detection.
- Building a change corpus with no offset no-change cases.
- Estimating heights from shadow in dense urban fabric.
- Evaluating a model on urban fabric different from the deployment.
- Relying on one metric where three levels of evaluation are needed.
- Using an existing reference without measuring its discrepancy against the imagery.
