Super-Resolution and Downscaling for EO and Earth Science
29-31 May 2024 | ESRIN | Frascati, Italy

Thanks to the recent rapid evolution of big data and image processing technologies as well as the availability of novel generations of Earth Observation (EO) satellites, vast amounts of EO data are now available for analysis for the benefit of science and real-life applications. The systematic acquisition of consistent EO datasets stimulate the development of new applications in the centre of potentially-viable long-term business opportunities. Additionally, the combination of the wealth of EO observations and sophisticated modelling techniques enhances our ability to comprehensively analyse and understand the Earth's systems, further advancing scientific research and supporting decision-making processes. 

In the field of EO, the interest in low-cost and open data for routine monitoring of the environment is growing exponentially in a wide range of applications. Important EO observations are available freely with very frequent temporal coverage, albeit at coarser resolutions than the typically commercial higher-resolution but lower-frequency offerings. Super-Resolution (SR) techniques could prove invaluable in this context by enhancing the spatial detail of lower-resolution imagery, bridging the gap between sensor generations and types. SR techniques can equally improve the spectral or temporal information of observations, with direct benefits for EO applications in areas where finer details are crucial for analysis and well-informed decision-making.  

In parallel, by employing downscaling techniques in various Earth Science disciplines (so-called SR in Computer Vision), more detailed information should lead to a better understanding of complex environmental processes in areas such as climatology, hydrology, ecology, weather forecasting, oceanography and air quality.  

With accelerated progress in computer vision mainly driven by Deep Learning (DL) approaches, SR and downscaling tasks have been implemented by a large variety of DL methods and techniques, ranging from the classical Convolutional Neural Networks (CNN) to more recent and promising SR methodologies using Generative Adversarial Nets (GAN) or Diffusion Models. Some of these techniques has sparked a new wave of Single-Frame Super-Resolution (SFSR) algorithms that enhance single images with impressive aesthetic results, but with potential non-existent details – so-called hallucinations. Multiple-Frame Super-Resolution (MFSR) can fuse series of original lower-resolution inputs into a composite higher-resolution image that can reveal some of the original detail that cannot be recovered from any of the lower-resolution images alone. 

Ensuring the resulting enhanced data is meaningful and validated for scientific or commercial use is of utmost importance. In the field of Computer Vision, assessing image quality and enhancement performance involve both subjective methods based on human perception and numerical metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). However, it is crucial to note that some Computer Vision metrics may not be directly applicable to EO data in particular or Earth Science data in general, where unique geospatial characteristics may require specialised and tailored metrics. Moreover, there is a critical need to balance between aesthetic considerations, the presence of artefacts, and the incorporation of new but accountable information. Achieving this balance is particularly challenging in EO and Earth Science, as processing techniques must enhance data without introducing misleading visual elements or compromising scientific accuracy and veracity.