True Gigapixel RTI
The ACRG has always been an integral part within the recent development of RTI. ACRG’s involvement began with the AHRC funded RTISAD project where we piloted the technique on inscribed ancient documents and archaeological artefacts. We likewise raised awareness of RTI in research and public communities in the UK. This has since led on to several organised community days such as the one held at Winchester Cathedral and the Re-reading the British Memorial community driven project that focuses on churches. The project has completed a number of these community days and is led by ACRG members Gareth Beale and Nicole Beale.
Since the RTISAD project our understanding of the technique has greatly changed. As new technology becomes available we are adapting our methodology to create new research avenues that not only help our department stand out but it also helps the wider community. Examples of this can be seen in the work completed by Eleni Kotoula who has created different methodological approaches in Microscopic RTI, Multispectral RTI and Transmitted RTI. Eleni, having come from a conservation background, saw potential in the technology within her own studies and many other members of ACRG are now utilising these methods as general practice within their own investigations. Further developments can be seen in the work completed by David Selmo who created the first underwater RTI dataset. He saw the potential that it had within maritime studies, developed the technique and then tested the methodology in open waters.
RTI has become an important tool within archaeological studies and at the ACRG we are always trying to develop this further. We have a number of examples and blog posts regarding this. My main personal issue however is that although our research has developed to introduce new varieties of RTI recording, our limitation has and will always be the resolution of the camera used. When RTI was first introduced by Tom Malzbender through his PTM viewer, the issue of resolution was not considered as the technology focussed on the changing surface detail and required no further detail. This has dramatically changed with the introduction of the technology within cultural heritage. Resolution is everything when considering images; the high the resolution the better, as minute details may make all the difference in fully understanding an artefact. This is especially true when studying pigment analysis. Eleni has considered this within her work by using microscopic RTI. This analysis however only works when you have small fragments where you know that the object recorded has potential in revealing information. What if I wanted to record a still standing Roman marble inscription that was once painted? I would only be able to record this by using the highlight method and as I have tried to explain, I would be limited by the resolution of the camera available.
The PTM fitter designed by Hewlett-Packard only has the capability to process a dataset that contains a height of around 4000px. This means that recent cameras that have been developed, such as a Nikon D800e (36mpx camera), cannot be used as the pixel height of a full image is 4912px. I have tried many times to process this dataset on different computers using the original ptm fitter and was left with the same memory error code. We purchased a Nikon D800e to enable us to further develop the technology and to allow us to examine the artefacts that we record in greater detail. As the PTM fitter could only process a limited number of pixels we were left having to reduce the resolution of the images in order to process them, which goes against the original idea behind its use. This however changed when I was introduced to a new ptm fitter developed by our ECS department, in collaboration with John Cupitt, which utilises an imaging program named VIPS. This PTM fitter runs via a command line but it enables users to process images that exceed the 4000px height. As a result of this we are now able to process the 36mpx images produced from the D800e and gain a higher resolution RTI dataset.
Having the ability to produce RTI datasets that contain higher resolution images, I began to consider the potential that this has within archaeology and my thoughts went back to the Gigapan RTI dataset that myself and Hembo Pagi captured in 2011 at Portus. The department had just bought a Gigapan Epic Pro and we were eager to try it out on site. Part of the work that we completed at Portus was the recording of panoramas and we had some success in producing high resolution images that far exceeded the capabilities of a normal camera. The system works by choosing a starting and finishing point, defining the available field of view, from which a series of automatic overlapping images are captured and then stitched together to produce a complete image. The camera that we used on site at Portus was a Nikon D3X (24mpx). As we were able to focus the individual images on small sections, it meant that the 100 or so images captured each contained 24mpx. When combined it meant that the overall image was highly detailed. As we had the equipment with us we decided to capture a RTI dataset using the same methodology used in the panoramic views. Portus has an abundant number of brick stamps and we carefully chose one that would enable us to capture the necessary dataset. We placed the Gigapan in front of the brick stamp, at a far enough distance away where lens distortion would not affect the stitching results. We took ten individual captures following the same methodology as used in a normal highlight capture. The Gigapan took a series of images where the light source was fixed. The camera had to move in order to capture the small detail but the Gigapan provides a memory function. After the Gigapan captured the automatic images we then moved the light source and simply replayed the last action of the Gigapan, making sure that we did not move the equipment in anyway. An example of the final rendered image can be seen below.
High resolution RTI dataset of a Portus Brick stamp
Although we captured this data in 2011 we were unable to process the ten test images until the VIPS PTM fitter was given to us, as we still had the same issue with processing images with a height of over 4000px. In order to produce these images, the Gigapan images had to be processed in a specific way. Kolor Autopano Giga was used to stitch the separate images but rather than batch process the Gigapan images automatically, I processed one and saved the control points. As different lighting positions were used it meant that certain RGB values would differ in each image and it could have affected the positioning of the images (if processed automatically) which in turn would affect the RTI dataset. Having a saved version of the control points that the software uses to stitch the dataset together meant that each separate light position image would be processed in exactly the same way and follows standard procedure within any RTI documentation.
Having now completed a test with these ten images my attention turned to completing a full RTI capture. I successfully managed to complete this several months ago using the same set up as we had at Portus. The below image shows the data capture in process with the Gigapan, object and shiny ball all being in a fixed position with only the light source truly moving.
Gigapan RTI set up for a 18th Century brick stamp
The item recorded within this follow up test capture was a brick stamp from the 18th century. The brick stamp belongs to Penny Copeland and was chosen because it has a great level of detail engraved into at the time of its firing. The brick I confess was recorded upside down but the intention of this data capture was to test if the VIPS PTM fitter could process a large number of images. The object was recorded using our Nikon D800e camera (36px) and a 200mm lens. The data capture took place in our dedicated imaging lab allowing us to have the camera set up far enough away from the object to avoid lens distortion problems. In total 42 seperate RTI images were captured and then processed. As I had used an even higher resolution camera and a lens that focused on very small detail, I was able to produce two separate rendered images using the same methodology discussed earlier. I was therefore able to produce images that could be used in a RTI dataset at resolutions of one gigapixel (one billion pixels) and two gigapixel (two billion pixels) and examples of these datasets can be seen below. These were then processed via the Cultural Heritage Imaging RTI builder in order to generate lp files and then run directly within the VIPS PTM fitter. This took some time to process because of the file size but we were able to produce PTM files of both. Having produced the datasets we encountered another problem in that we could not open the files as the standard RTI viewer runs out of virtual memory. The one gigapixel RTI file size is 8.52gb! Although this is problematic, I am currently working with the University’s supercomputer department, Iridis, in order to create a GUI that will enable us to open the dataset and the ACRG are currently working on developing a new web based RTI viewer that will enable online sharing of these datasets. Once this has been completed I will add a follow up blog post showing the RTI rendered images.
One gigapixel RTI dataset. To view in full screen please visit the iip server where it is hosted
Two gigapixel RTI dataset.
In the mean time discovering that we are now able to process large datasets means that we can rethink our approach to how we use RTI within Cultural Heritage. The image resolution is always hard to calculate and this is especially true when considering the dataset that we have. Normally the number of lines per inch a camera can resolve at the sensor equates to the resolution of the image produced. Within our own data we are not using a single sensor but rather a sensory array so any calculation of resolution must be based as an expression of pixels at a particular point of the image. In the two gigapixel dataset we are able to calculate, using the scale provided within the image, that 1cm in the real world is represented by (is sampled into) 890 pixels. Therefore each pixel is 1/890 cm = 0.001123cm, Meaning that a pixel equates to 11 microns resolution. This resolution, if correct, means that we were able to capture and process a RTI dataset at a microscopic resolution without the need of a microscope and highlights the potential that this has within specific artefact studies. Of course this technique will not replace microscopic analysis but rather allow users to view larger areas using the same RTI analysis as usual, but then allow them to also have the microscopic detail if required.
The basic technique of this high resolution RTI data capture is still under development and much of the software is still being updated to incorporate the need for higher resolution images. At the ACRG we have taken this one stage further by producing a true gigapixel RTI dataset. This is of course an extreme and for most studies is not needed. However the technology and methodology now exists that will enable future researches to use this technique in whatever way they feel and with whatever image size they capture their data with. For those debating whether or not this next step in RTI is worthwhile, I will go back to my earlier question regarding the standing Roman marble inscription. What method should I use? Normal highlight RTI, Microscopic RTI or Gigapixel RTI? I will let you decide!
