Putting the science in fiction - Dan Koboldt, Chuck Wendig 2018
Imaging over long distances
Sometimes, it really is rocket science
By Judy L. Mohr
The satellite whizzes overhead, its alignment reconfigured by the technician in some bunker in a secret location. After moments of clicking at the keyboard, a series of images flicker across the screen. Details of the landscape come into focus, but that detail is not enough. The technician taps the keyboard again, clicks the mouse, and the cameras on the satellite overhead zoom in. They’ve found him. They can see exactly what he’s wearing and the backpack slung across his shoulder. Oh no … The hero is now in danger. Run, sly spy! Run!
While Hollywood would in reality take those zooming-in shots using a hover drone, believe it or not, the concept that the moviemakers are trying to portray is very real. As much as you might try and hide, you can’t: The spy satellite will see you.
It’s not a simple matter of just taking a picture like you would on your cell phone or digital camera. In reality, there would be a series of images captured, in conjunction with behind-the-scenes algorithms and corrective optics to compensate for atmospheric effects in real time and during post-processing.
Mirages and rippled glass
Most people have firsthand experience with the effects that the atmosphere has on light. Think about driving down a long, straight road on a sunny day. In the distance, you see shimmering above the road, and what some might believe is a wet patch. However, the closer you get to the origins of the mirage, the effects diminish and practically disappear. Instead, the wet patch has moved to a location farther into the distance. This mirage effect is caused by heat rising off the ground, but the effects that air movement has on light need distance to propagate before our eyes can detect it. The farther into the distance you look, the greater the scintillation. It’s because of this effect that the stars twinkle at night.
Within the field of optics, one can think of the atmosphere as a thick pane of glass, but not just any glass. I’m talking about that rippled glass commonly used in bathrooms. If you were to shine a light through that glass, you would get a rippled pattern on the wall behind it where some parts are brighter than others. Now imagine that the rippled pattern in the glass was constantly moving. The bright sections of light would be flickering—they would be twinkling.
Imagine trying to take a picture of something that is constantly moving in an erratic way. For short-exposure images, the resulting speckle pattern can make it difficult to identify the shape of the original object (see Figure 48.1). For long-exposure images, you get a blurry mess. It might turn out to be quite an artistic image, but for someone on the ground trying to take pictures of the stars, or a satellite trying to take surveillance images of the ground ... Not a good thing.
Figure 48.1: The effect of the Earth’s atmosphere on stellar light. (a) A simulated stellar image taken from space (no distortions introduced by the atmosphere). (b) The real situation where a speckle pattern is seen in the short-exposure image due to the aberrations induced by the atmosphere. [Source: Mohr (2009). “Atmospheric Turbulence Characterisations Using Scintillation Detection and Ranging.” Ph.D. Thesis, University of Canterbury.]
In reality, the atmosphere is not a single sheet of glass with a moving rippled pattern. It’s actually multiple sheets of glass, each with different characteristics. In some cases, that layered sheet of glass is incredibly thick. However, if you were able to determine exactly how the layered sheets in the atmosphere were rippled and how they were changing with time, you could compensate for the resulting twinkling effects.
Surveillance imaging in the desert
Advancements in digital imaging are progressing every day. The latest smartphones now use 16 megapixel cameras and digital zoom to narrow in on that long-distance detail.
However, those spy films often show the hero in the desert of the Middle East taking images of the target from miles away. Over those distances, the scintillation would make a standard image so fuzzy that detail would be hard to distinguish. Some people might argue that if you use a camera with a telescopic lens, you’ll easily take the detailed image you require—but it’s not that simple.
Even if you had the best camera in the world with the most expensive optics mounted on front, unless you employed some level of post processing the resulting image would still be fuzzy. However, like in the stellar situation (Figure 48.1), if you could model the effects that the atmosphere has on the light, you can compensate for it. The equations for the horizontal problem are significantly more complicated, but it is doable.
In fact, in 2002, researchers from ADFA (Australian Defence Force Academy) in Canberra, Australia presented to the world single-frame images obtained of a house located ten kilometers away from their imaging system (over six miles). If there had been a person standing at the window at the time the photos were taken, you would have seen them in those photos (see Figure 48.2).
Figure 48.2: Single-frame image from a telescope sequence obtained over a horizontal distance of ten kilometers (over six miles). [Source: Jahromi, et al. (2002). “Image Restoration of Images Obtained by Near-Horizontal Imaging through the Atmosphere.” DICTA2002: Digital Image Computing Techniques and Applications, 21—22 January 2002, Melbourne, Australia.]
If you combine the technology developed for imaging over long-distances from 2002 with the digital camera technology of today … Let’s just say that I have no problems in imagining that surveillance photos could easily be taken of me, and I would never know it.
Long-distance imaging available to all
Imaging over long distances has been an area of research for many years. Granted, large portions of the research have been initiated by military and spy-related activities, but other sectors also benefit from the technology: astronomy (which is how I came to be in this field myself), telecommunications, civil aviation, and users of Google Earth, just to name a few.
You read that correctly, folks: Google Earth has taken advantage of satellite imagery for years. In 2007, Google Earth undertook a massive campaign to image my home city, Christchurch, New Zealand. The initial images blew everyone away, using a combination of satellite imagery, aerial photography, and geographic information systems. Since then, Christchurch has suffered massive earthquakes, leveling whole areas of the city, including the central business district. All of those initial Google Earth images are now not accurate enough to use. New images have been taken, but most of the residential areas are now cheap, minimal-rendering, satellite images. Even so, the detail in the images is impressive.
Take the image of my own house and surrounding neighborhood (see Figure 48.3). Due to copyright issues, this particular image was not acquired from Google Earth, but rather a company in the United States that specializes in satellite images for commercial use. While the level of detail visible in this image is better than the cheap, minimal-rendering images currently used on Google Earth, the image itself is similar in quality to the original images from 2007, but with post-quake details.
Figure 48.3: Satellite image of my house and surrounding neighborhood.
You can clearly see my big red roof (the only rectangular house), complete with the chimney flu and the white vented port by the back door. My SUV is in the driveway, and the garbage wheelie bins are by the fence. There are all the trees on my property, and the neighbor’s patio with the umbrella covering her picnic table.
This image was taken using a satellite that is years old. Just imagine what information can be gleaned if you actually took the time to focus and acquire a series of images using the latest imaging technology.
Given the public-domain research available in this field, most of which is years old, it’s no stretch of the imagination to believe that a satellite could take a photo of you and determine whether you were holding a tennis ball or a basketball. However, that spy agency is going to need ground-based surveillance footage if they want to see you peel that orange.