Automated data gathering and display

A scientist was conducting research that depended on three sources of data: a reference data set; a set of observations that updated weekly; and a set of observations that updated daily for part of the year. The scientist would retrieve the observations from two different website accounts, perform significant manual processing to juxtapose comparable data across all three sources, and make decisions whether to adjust the course of study.

We helped the scientist by automating most of these steps. We wrote Internet robots to retrieve and process the observations at the appropriate times. We wrote queries to show comparable data in priority order. We set up a private area on our company's collaboration server, so that the scientist could view the work and make decisions at will, whether at the office or at home. The scientist was happy with the result.