The Ionia 1-km Net-Browser Experience
Quicklook Processing and Access Statistics

J.-M. Melinotte & O. Arino

ESA/ESRIN, Projects & Engineering Dept., Earth Observation Division, Frascati, Italy
Phone: +39-6-941 80 564 Telefax: +39-6-941 80 280

The number of 'Previews' (mini Quicklooks) and Quicklooks transmitted by network is increasing continuously: more than 30 000 'Previews' have been downloaded during the month of May 1995. More than 80% of the overall traffic is generated by less than 30% of the users. European users represent 60% of the users. The service to the operational, application and research communities represent more than 65% of the overall traffic. These 65% are shared by relatively few operational and application entities and a myriad of research centres. The Ionia 1-km experience could help to define the network Browse services for future Earth- observation systems like Metop and Envisat.

INTRODUCTION

Ionia is the name of a series of tools designed to give the visibility of the low-resolution sensors (e.g. AVHRR) datasets archived and maintained at ESA/ESRIN (Arino, 1993).

The Net-Browser package consists of a commercial WWW HTTP server with additional features for inventory searching, image display and product selection. The system design takes into account the experience of the AVHRR CD-Browser Ionia (Arino et al., 1993, Melinotte & Arino, 1993).

The 1-km AVHRR Global Land Dataset project is an international effort to acquire, archive, process and distribute the data of the entire global land surface observed by the NOAA/ AVHRR satellite mission at 1-km resolution (Eidenshink & Fraunden, 1994). It meets the needs of the international science community (IGBP, 1992, Towshend et al., 1992). A cooperation plan was established between NOAA, NASA, the US Geological Survey (USGS), ESA and the Common- wealth Scientific Industrial Research Organisation (CSIRO) with the aim to ensure the completion and the coordination of this project. A network of 29 high-resolution picture transmission (HRPT) stations, along with data recorded by NOAA, has been acquiring daily global land coverage from the NOAA-11/14 satellites since 1st April 1992. The continuous flow of data coming from these stations is routinely archived by USGS and ESA/ESRIN.

The success of the project is assured by the work performed by the teams working at all ground receiving stations. These are:

• Coordinated by USGS and CSIRO: Bangkok, Buenos Aires, Beijing, Casey, Darwin, Dharan, Guangzhou, GSFC Washington, Hartebeestoek, Hobart, Uh Snug Harbor, Jeddah, Louisiana, Macmurdo, Fairbanks, Wallops Island, Prince Albert, Palmer Station, Perth, Ryadh, Sioux Falls, Syowa, Tokyo, Towsville, Urumqi.

• Coordinated by ESA/ESRIN: Cairo, Frascati, Maspalomas, Niamey, Nairobi, Oberpfaffenhofen, La Réunion, Tromsø, Terranova, Manila, Scanzano.

THE IONIA 1-km QUICKLOOK

The new digital Ionia Quicklook algorithm has been designed and developed in order to:

• give a good visual assessment of the corresponding AVHRR product quality and characteristics, for instance: image location, data quality (bad lines), cloud coverage, characteristics of the sensed surfaces (e.g. cloud height, water temperature, sun glint, vegetation strength, ice and snow);
• be accurate for any AVHRR type of instrument at any latitude and acquisition time. It will easily work with the ERS-2/ATSR-2, Envisat/ AATSR, EOS/MODIS and ADEOS/ OCTS.

This is achieved by calibrating all input data and then performing histogram equalisation on pre-defined segments as described hereafter.

Input data and calibration
For generating the 1-km Quicklooks, all HRPT passes have been processed to level 1B. This allows to access to all view and solar irradiance angles as well as infrared calibration coefficients. It was therefore possible to convert raw data into reflectance for channels 1+2 using the calibration coefficients provided by Kaufman & Holben, 1993, in brightness temperatures for channel 4 using the in-flight calibration coefficients as described in Lauritson & al., 1979. Calibration allows the processing to be independent for any geo- location (from South pole to North pole) and for any time as well as for any of the NOAA satellites carrying an AVHRR.

Colour assignment
The Ionia Quicklook is a 3-colour composite of channels 1+2 reflectance and channel 4 brightness temperature. Each channel is decomposed by surface class, providing a continuous and coherent colour spectrum spanning from 0 to 255 numerical counts. Specific classes like vegetation were enhanced by increasing their representation in the colour spectrum as shown in Figure 1. After colour attribution, an histogram equalisation is performed separately on each single segment. Therefore ten histogram equalisations are performed for each single Quicklook. This technique allows the visual enhancement of interesting features even when the class is underrepresented in the image. The colour restitution in the Red- Green-Blue components of different target is shown in Figure 2. As a reference, the spectral responses of sand, vegetation and water in the visible and near infrared spectrum are given.

Figure 1. Colour look-up table

Figure 2. Colour restitution mechanism

Spatial resolution
A sampling procedure is applied along and across track to achieve a constant pixel size: 6 km x 6 km.

Compression
The average Quicklook size is reduced by a factor 15 using JPEG format (~ 80 kbytes for a 15-minute pass), thus facilitating the distribution to the users on the network.

Preview generation
The previews are a kind of 'thumbnails' of the original Quicklooks, they are generated by applying a 1/4 spatial sub-sampling resulting in a pixel size of 24 km and by reducing to 64 colours. With their 10-kbyte size, the previews are easily transmitted via network.

THE IONIA 1-km NET-BROWSER

The experimental Ionia 1- km Net-Browser uses public domain systems such as the networked browser NCSA Mosaic (Andreessen, 1993) and the Netscape. The core of the NCSA Mosaic and Netscape systems are based on the World- Wide Web, a distributed hypertext-based information system developed at CERN. As such NCSA Mosaic and Netscape speak a lot of common TCP/IP network protocols, including HTTP, Gopher, FTP, and NNTP. The Quicklooks, Previews and Inventory informations are integrated within NCSA's Mosaic and Netscape allowing their retrieval by simple click on the underlined word or phrase which is a hyperlink to another document anywhere in the network. Both systems support the GIF and JPEG image formats as well as other standard formats. Ionia 1-km Net-Browser Internet address is:

http://shark1.esrin.esa.it

After reading the Home Page and other information pages a normal 'Browse' session starts by defining the search criteria either geographical (clicking on the world map) or temporal (defining the start and stop dates) in the Query Page. In a few seconds, the server returns the list of items retrieved, chronologically sorted and grouped by ten. Each group of items can be viewed separately, the server then displays the corresponding Previews and Inventory information inside the Preview Page. From this page, other possibilities are offered such as product selection (for ordering), full Quicklook downloading, map location downloading. When a product selection has been performed, the Order Page must be loaded in order to fulfil the form and to send or print the final order.

THE IONIA 1-km STATISTICS

The access statistics after one year of experiment are given hereafter. The Ionia 1-km Net-Browser was open to external world on 17 May 1994. The Ionia AVHRR Quicklooks were gradually introduced into the system: the service has been open with approximately 3300 Quicklooks and counts now more than 15 000 Quicklooks on-line. In one year, more than 130 000 Quicklooks have been down-loaded by more than 13 500 users.

Figure 4 shows, for each month, the number of queries and Quicklooks (previews and full quicklooks) retrieved by the external users: apart from the first peak (May 94), mainly due to the advertising on NSCA mosaic 'What's New' page, it clearly appears that the number of accesses and transfers increase regularly. The tendency to retrieve Quicklooks from the server is directly linked to the amount of data available on line. More than 30 000 Quicklooks have been retrieved for the ultimate month analysed (May 1995).

Figure 4. Quicklooks retrieval monthly statistics

Figure 5 shows the result of a simple analysis concerning the type of users accessing the Ionia 1-km Net-Browser. Around 80% of the traffic is generated by users performing effective professional browse activity (we consider an effective professional browse activitie a single machine performing more than 30 calls). This 'professional' interest comes from very few centres like for instance: INRA, JRC, Meteo France, DLR, CNES, Ericsson, NRSC, NERC, RAL, SSC, CNR, INRIA, representing around 30% of the global users. It has to be noted than the net surfers despite representing more than 60% of the overall users do only account for less than 13% of the traffic (the net surfers are the one having generated less than 20 calls: this means that they stop the session before receiving a complete 'Preview Page').

Figure 5. Curiosity versus professional use

Fig. 6b

Fig. 6a+b have been generated from the statistics the users considered as 'net surfers', so keeping only those users having performed a real browse session (having transferred at least 10 Previews). A quick analysis of Figure 6a shows that:

• Around 56% of the traffic is generated by the European community users;
• 20% by the US (10% education and 10% commercial);
• research, operational and application entities represent 86% of the traffic.

The split between research, operational, application and commercial users is difficult to establish. A sorting has been tentatively performed for the French users and is given in Figure 6b:

• School, Universities, CNRS, Observatories and computer centres have been counted as research. They represent the more numerous community with 55% of the users and 39% of the Quicklooks traffic.
• CEA, CNES, Meteo-France, CNRM, DGAC, Ifremer, CNET and IGN have been classified as operational entities and represent 29% of the Quicklooks traffic concentrated in a few operators.
• ORSTOM, INRA, CIRAD, Cemagref, Engref have been classified with the application users and represent 21% of the traffic also concentrated in a few centres.
• Some commercial entities (mainly in communication, space and computer fields) identified by their name generated a limited traffic of around 3%. This is not surprising for low-resolution data such as the AVHRR with little commercial value.
• We were unable to classify 11% of the customers, while some of them use to download an important quantity of Quicklooks

CONCLUSION

Ionia 1-km Net-Browser gives the Internet community the possibility to browse through the inventory information and the Quicklooks generated by ESA/ESRIN for the 'Global Land 1-km AVHRR Dataset'. A specific colour digital Quicklook has been developed for Network distribution. The processing technique is generic and could be applied to any AVHRR-like sensor: ATSR, SeaWiFS, MODIS, MERIS. The experimentation of the Ionia '1-km' Net-Browser for more than one year, allows to draw the following conclusions:

• The potential use of the Internet capacity to broadcast small Earth observation datasets like the Quicklook products has been successfully demonstrated.

• The need of an integrated system (e.g. Quicklooks dimensioned to the Internet system capacity) is manda-tory for a successful usage.

• Local Users (European) represent the main budget of the users.

• Few users (operational and application driven) do represent the major part of the data traffic.

• Curious do not overload the Network but shall be accounted for.

ACKNOWLEDGMENTS

The authors wish to thank the following teams: Z. Bjelogrlic, F. Mungo & G. Serrecchia from Intecs Sistemi; A. Buongiorno, E. Monjoux & G. Pittella from ESA/ESRIN, for their work, inputs or discussions concerning the subject.

REFERENCES

Andreessen M, 1993: NCSA Mosaic Technical Summary. Revision 2.1., 4 May 4 1993, National Center for Super-computing Applications, 605 E. Springfield, Champaign, IL 61820.

Arino O, 1993: AVHRR data acquisition processing and distribution at ESA. Eurocourses on Advances in the use of AVHRR data for land application, Ispra, 22-26 Nov. 1993.

Arino O, J-M Melinotte & G Calabresi, 1993: Fire, cloud, land, water: the 'Ionia' AVHRR CD- Browser of ESRIN. EOQ, 41, July 1993.

Eidenshink J C & J L Faunden, 1994: The 1-km resolution global dataset: Current progress. IJRS, 15, 3443-3462.

IGBP Report No. 20, 1992: Improved Global Data for Land Applications. Report of the Land Working Group of IGBP-DIS. Edited by J.R.G. Townsend.

Kaufman Y J & B J Holben, 1993: Calibration of the AVHRR visible and near-IR bands by atmospheric scattering, ocean glint and desert reflection. Intl Jl of Remote Sensing, 14, 21-52.

Lauritson L, G L Nelson & F W Porto, 1979: Data extraction and calibration of TIROS-N/NOAAA radiometers. NOAA Tech. Memorandum NESS 107, Washington D.C.

Melinotte J-M & O Arino, 1993: AVHRR CD-Browser 'Ionia'. Proc. 6th European AVHRR Data Users Meeting, Belgirate, 29 June-2 July 1993.

Mungo F, O Arino, J-M Melinotte & Z Bjelogrlic, 1994: Ionia '1-km' Net-Browser. Proc. 6th Intl Symp. on Physical Measurements and Signatures in Remote Sensing, ISPRS, Val d'Isére, 17-21 January 1994.

Townsend JRG et al., 1992: The global 1-km AVHRR Dataset: Further Recommendation. IGBP-DIS Working paper N. 3, 10 June 1992.Arino O, 1993: AVHRR data acquisition processing and distribution at ESA. Eurocourses on Advances in the use of AVHRR data for land application, Ispra, 22-26 Nov. 1993. Arino O, J-M Melinotte & G Calabresi, 1993: Fire, cloud, land, water: the 'Ionia' AVHRR CD-Browser of ESRIN. EOQ, 41, July 1993.