Aerial photography and ground obscuration (part 3)

I have previously written twice about factors affecting the possibility of discovering archaeological features using aerial photography (1) (2).  The original version of this took into account human factors (see post 1), based upon OS OpenData, specifically buildings, roads, railways, and woodlands / rivers / lakes (I count woodland as a human factor, as many of them are plantations and others have been left / managed by humans, rivers / lakes are obviously less human influenced).  Here is a reworked example, showing the percentage of the ground surface obscured by such factors:

Ground obscuration taking into account woodland and human structures. Relevant to aerial photography generally.

To add to this previous work, a few months back at our Project Advisory Board meeting, Jeremy Taylor of the University of Leicester told us about a paper which classified the different soil types of England and Wales according to their prospects for showing buried features (whether geological or archaeological) as crop marks (Evans 1990).  This paper grouped the different soil types defined by the 1983 Soil Survey of England and Wales into five categories:

  • Soils that show extensive crop marks;
  • Soils that show extensive crop marks in dry conditions;
  • Soils that show frequent crop marks over small areas;
  • Soils that rarely show crop marks;
  • Soils that never show crop marks.

Usefully, the 1983 soil survey maps have been digitised and are made available via the National Soil Resources Institute (NSRI) at Cranfield University.  As a bona fide researcher, this is available on payment of a processing fee (i.e. without paying royalties), so we obtained this data to try to create a map of Evans 1990 classification.  On receipt, I found that there were five soil types (924a, 924b, 952, 961, 962) in the NSRI data that did not show up in the Evans 1990 classification.  However, these were all types of industrial spoil heap or reconstructed ground surface, so I assumed that they would fall within the category of soils that never show crop marks.  As a result, I was able to reclassify the NSRI soils data into Evans’ five types:

Soil types of England and Wales graded according to possibility of showing crop marks. After Evans 1990.

In respect of ground obscuration, I decided to only take the “never” category as being a factor.  I therefore combined this with the “human” factors and the peat / alluvial sub-soils from the British Geological Survey data (see post 2) to create a new map of ground obscuration:

Ground obscuration taking into account soil type, some sub-surface geology (peat / alluvium), woodland, and human structures. Relevant to crop marks.

This latest map is only really relevant to crop marks, as presumably earthworks would still show even in the more unsuitable soils (under the right lighting / weather conditions).  We can see, however, that large parts of England are not at all suitable for aerial prospection for crop marks.  I am not sure that the large peaty areas seen in the east would necessarily mask all crop marks, but then all models are imperfect.  It might be useful to include the “rare” crop mark soils in this map too, perhaps with a lower weighting to reflect their partial effect, but I decided to work for now on a stricter basis.

The point of all this is to assess the effect of ground obscuration on the patterning of archaeological discoveries.  As an example, we can compare the distribution of NMP projects against this map of ground obscuration:

NMP projects as of 2012 compared against ground obscuration (re. crop marks).

This shows a reasonably good correlation between areas in which NMP projects have been (or are being) undertaken and areas suitable for aerial prospection (for crop mark features).  Where the NMP projects cover areas unsuitable for crop mark discovery, these tend to be upland areas where thin soils ought to mean earthworks are fairly prominent (I think).  It also shows that although the NMP covers around 50% of England, much of the remaining 50% is not necessarily suitable for NMP type projects (as they are reliant on aerial photography), with the particular exception of the rural West Midlands and parts of East Anglia.  However, many of these less suitable areas might be suitable for LiDAR survey.

Further down the line, I will be comparing distributions of archaeological sites against these obscuration maps to see if I can discover any intrinsic biases towards areas suitable for aerial prospection within the data.

Chris Green


Evans, R.  1990.  “Crop patterns recorded on aerial photographs of England and Wales: their type, extent and agricultural implications.”  Journal of Agricultural Science, Cambridge 115, 369-382.

Processing raster NMP tiles

One of the most important datasets with which we are working on this project is the English Heritage (EH) administered National Mapping Programme (NMP), which has been working all across England to map archaeological features seen in aerial photographs.  This project has been ongoing for about 20 years and, as a result, the methodologies used by those undertaking the work has evolved over time.  More recent NMP projects were undertaken in CAD software, with the result that the drawings produced exist as vector data, which is very flexible for GIS mapping purposes.

However, I have been thinking over the last few days about how to deal with the older areas of the NMP that exist as scanned raster images of line drawings.  This raster NMP data is less flexible, in that raster imagery does not scale well when looking at broader regional patterns (when compared to vector data) and it is difficult to display any background data behind raster tiles (making the white areas invisible tends to make the black lines fade, as obviously does making the layer partly transparent).  I have also been talking recently via email with Helen Wickstead about some of her ideas for examining the topological relations in field system layouts and this type of analysis would be impossible with raster scanned maps.

1 raster NMP
Example of raster NMP tiles

As a result, I decided to try to convert the raster NMP tiles received from EH to date into vector data.  I came up with quite a simple system to do so, after a bit of trial and error.  First, we reclassify the data in the raster tile so that the white areas have a value of ‘NoData’ and the black areas (i.e. the drawings) retain a value of 1 (for most tiles, the black areas already have a value of 1 and the white of 0, but this is reversed for most of Kent and some tiles in Lincolnshire).  The next stage is then to use the Raster to Polygons tool to convert this result into a polygonal vector layer.  Ideally we would want to convert them to lines rather than polygons, but they are too ‘thick’ for this to work (more on which later).

The result includes a large amount of small polygons which add little to the overall picture other than making the layer render on screen slowly.  If we assume that these original drawings, which were done at a scale of 1:10,000, were undertaken with a minimum pen width of 0.1mm, then any polygon smaller than 3.14m² would be smaller than the head of a pen and, as such, unlikely to be deliberately plotted.  Therefore, we then calculate the area of the polygons in the layer, select those with an area of greater than 3m² and export that as our final result.  For the datasets processed to date, these small polygons amounted to about 20% of the total, which have been removed to no obvious visual effect.

Obviously, this is a lot of work to undertake when dealing with large sets of 5 x 5 km tiles for large swathes of England.  As such, I experimented with the Model Builder in ArcGIS to see if I could automate the process.  The process had to be split into two stages, due to the software getting confused by the new file names.  Stage 1 iterates through a folder of raster layers, reclassifies them and then converts them to polygons:

2 model stage 1
Stage 1 of the model to convert raster NMP tiles to vector data

For some reason, the Model Builder was not attaching projection information to the output layers, despite the fact that I had specified this under the tool’s instructions.  The projection needs to be defined to accurately calculate the area of extent of the polygons.  As such, Stage 2 iterates through the data output by Stage 1, defines the projection for each layer as British National Grid, and then calculates the area of extent of each polygon:

3 model stage 2
Stage 2 of the model to convert raster NMP tiles to vector data

We then merge all of the layers output into a new composite layer, use the Select by Attribute tool to select all of those polygons with an area of more than 3m² and export the results.  Here we can see the vectorised result for the area of raster NMP shown above:

4 vectorised result
Vectorised version of raster NMP tiles

I am reasonably satisfied with this result, but there are a few things I would still like to do to tidy up the layer.  That is to remove the grid marks (or grid lines in the case of some NMP projects, e.g. Dartmoor and most of Kent) and, ideally, to remove the pieces of text on some of the images (particularly Dartmoor).  I cannot (yet) think of an automated way to deal with the latter, but removing the grid marks / lines can be automated, albeit with a somewhat imperfect result.

Removing grid marks: input data

We begin with a layer of grid lines that fall on each of the 1000m points east and north of the origin of the British National Grid, created using the Geospatial Modelling Environment software:

Removing grid marks: 1000m grid lines

We then create buffers 5m either side of these grid lines (i.e of 10m width).  This covers most of each grid mark / line (for areas of raster NMP that include drawn grid lines, the next two steps in the process are not needed, we would skip straight to the erasing):

Removing grid marks: 5m buffers each side of grid lines (i.e. 10m total width)

We then create 45m circular buffers around the vertices between these grid lines:

Removing grid marks: 45m buffers around grid line vertices

Then we clip the line buffers by the vertex buffers to create a polygon layer that masks the grid marks:

Removing grid marks: grid line buffers clipped to buffers around vertices to produce cross mark mask

Finally, we use the Erase tool in ArcGIS to remove the areas of the vectorised raster NMP layer covered by the grid mark (or line) mask (at this stage, I also clipped the layers to the coastline for coastal areas):

Removing grid marks: the result after erasing polygons under cross mark mask

The result is clearly imperfect, but does look cleaner than the original result.  For some areas, bits of grid mark or line still appear within the resulting layer, where they slipped outside the mask, but these can be cleaned up manually where necessary.  The main problem with this automated approach is that it can cut out small areas of the drawings that relate to archaeological data where they overlap grid marks / lines, but these should be fairly minimal, and largely invisible at broader scales of rendering.

6 grid removed
Vectorised version of raster NMP tiles with grid marks removed

By way of comparison, here is an area where vector and raster NMP data meet:

7 compare vector & raster
Comparison of vectorised raster NMP tiles (in black) against vector data (coloured)

We can see that the vector NMP areas have more attribute data attached to them, so that we can plot this data in different colours according to the morphology of the feature (i.e. bank, ditch, etc.), but the vectorised raster NMP areas are now easier to plot alongside the vector areas, especially on these type of broader landscape scales.

I am quite pleased with this result, and it certainly helps greatly with the plotting of the areas of the NMP which exist as raster data, but it does not help greatly with the previously suggested idea of examining the topological relationships within field systems.  I think that there may be a way to produce a line (rather than polygon) result by processing the original data using the Thin algorithm in GRASS GIS, but this would need care and is probably only feasible for quite restricted areas.  I shall return to this idea at a later date!

Chris Green