Software Tools for both Earth and Social Sciences

Software has obviously been an important component of both scientific research and engineering applications for several decades, as well as an increasing presence in everyone's life (e.g., your mobile phone or PDA, your car, etc). The big problems of our time have spread across disciplines, and require the collection, management, and analysis of large amounts of data by numerous groups and individual researchers. One of the attempts to address this issue is the use of Grid Computing to formally share both data and computing resources, along with several software frameworks to access and analyze grid-based resources and even run complex scientific workflows (see the Kepler Project below).

An important point to note is that these problems and related research areas go across disciplines, from earth science to sociology, and from ecology to public policy. The links between formerly disparate fields continue to grow and strengthen, making the understanding and use of appropriate software tools of critical importance for everyone. Many fields have become intertwined in interesting ways, as the following two examples illustrate.

1. The National Center for Ecological Analysis and Synthesis (NCEAS, a research group affiliated with UCSB) brings together researchers, post-docs, and students to work on cross-cutting interdisciplinary research that uses existing data to address major fundamental issues in ecology and allied fields, and their application to management and policy. This work requires the development and use of new tools that not only provide the capability to look at existing data in new ways, but also support users who come from fields perhaps previously considered non-technical (or at least not usually known for being extremely geeky :). The research at NCEAS has evolved into a discipline in its own right called Ecoinformatics, and they've developed Kepler (see below) along with some related tools to help facilitate this work. NCEAS’ Ecoinformatics Program is dedicated to the research, development and dissemination of technological tools that facilitate analysis and synthesis in ecology and the environmental sciences.
2. The Cognitive Science program at UCSD studies areas that cut across the brain, behavior, and computation. They have several undergraduate degree options that require significant work with both applied math and software tools (including software development). Some of the options are actually cutting-edge computing degrees masquerading in what may look like the Psychology Dept at first glance (but really isn't). For example, required topics for both undergraduates and graduates include courses in behavior, computation, and the neurobiological basis of cognition. And yes, they use Python...

But even basic problems in geography and related social sciences require the use of geospatial and mathematical tools to work with large amounts of data with spatial attributes, as well as flexible automated text processing. Web-enabled applications are frequently useful, and there are many other examples below. Besides the Zope web application framework and Python, there are many useful software packages for users in all these fields, for everything from data ingest and transport to analysis and visualization, whether on the web or the desktop. Another important point to note in this case, as that these kinds of tools are not just for geeky types, science or otherwise. Python is a prime example of a powerful tool that was designed for normal humans (as well as geeks :) Check out some of the free books and some example Python code if you don't believe me (especially if you're a non-programmer, you should read this).

Online Python Books and Tutorials

• How to Think Like a Computer Scientist: Learning with Python - The original book is still available, and it looks Allen Downey has a revised book called How to Think Like a (Python) Programmer (both are available freely online or can be purchased in printed form).
• Alan Gauld's Learning to Program - Another free Python book on the web that's also available in print; it looks like the book is still stuck on Python 1.5, but the web site is much more current.
• Official Python Tutorial - This is the one written by Guido, the creator of Python (a nice Python API reference for multiple versions is available here).
• Text processing in Python - Another free Python book from David Mertz, also available in printed form; I guess it seems oddly appropriate that this one comes as a set of ASCII text files...
• For more advanced users, there's Dive Into Python, and Bruce Eckle's draft version of Thinking In Python is also a good choice.

Scientific Modeling Frameworks

• Ptolemy II - Ptolemy II is a Java-based component assembly framework with a graphical user interface called Vergil. The Ptolemy project studies modeling, simulation, and design of concurrent, real-time, embedded systems. Ptolemy II's generic framework is flexible enough to be used for many different problem domains from discrete event simulation and modeling of communication networks, to scientific data analysis workflow tools such as Kepler (see below).
• Kepler - The Kepler project is a cross-project collaboration to develop open source tools for Scientific Workflows (partly based on the Ptolemy II framework).

General Scientific Software

• Scientific Python - SciPy is an open source library of scientific tools for Python. SciPy supplements the popular Numeric module, gathering a variety of high level science and engineering modules together as a single package.
• Octave - GNU Octave is a high-level language, primarily intended for numerical computations (compatible with Matlab).
• R - The R Project for Statistical Computing, a free software environment for statistical computing and graphics (R is essentially the GNU implementation of the original S language and environment). Note there are several stats textbooks written around R.
• HDF4 / HDF5 - A general purpose library and file format for storing scientific data.
• NetCDF - NetCDF (network Common Data Form) is an interface for array-oriented data access and a library that provides an implementation of the interface.
• UDUnits - A library for manipulating units of physical quantities.

GIS, Mapping, and Geospatial Data Analyis

• GRASS - GRASS GIS (Geographic Resources Analysis Support System) is an open source, Free Software Geographical Information System (GIS) with raster, topological vector, image processing, and graphics production functionality.
• UMN MapServer - A popular web mapping application with support for GDAL, proj, and other tools.
• ZMapServer - a Zope interface to mapserver; currently being re-factored into Cartographic Objects for Zope and the Python Cartographic Library.
• OpenLayers - An open source pure JavaScript library for displaying map data in most modern web browsers, with no server-side dependencies. Use hobu's example application to browse the state of Iowa.
• Proj.4 - Proj.4 cartographic projection software (additional grids, eg, NAD27 available).
• GDAL (Geospatial Data Abstraction Library) - a translator library for raster geospatial data formats.
• OGR Simple Feature Library - OGR is a C++ library for a variety of vector file formats (also has C and Python APIs).
• OpenEV - An OpenGL/GTK/Python based graphical viewer which uses GDAL for raster access.
• Shapelib - A library for working with ESRI shapefiles (ArcInfo compatible).
• Virtual Terrain Project - a set of software tools, including an interactive runtime environment (VTP Enviro). The VTP libraries also use GDAL.
• gstat - a geostatistical modeling package.

Meteorological Modeling and Data Analysis Software

• LAPS - LAPS Local Analysis and Prediction System, a real-time meteorological analysis package. You must sign up on the list and request the software.
• MM5 - Penn State/NCAR Community Mesoscale Modeling system (needs PG Fortran compiler on Linux).
• LDM - The Unidata Local Data Manager (LDM) is a collection of cooperating programs that select, capture, manage, and distribute meteorological data products.
• GSD - NOAA's Earth System Research Laboratory's Global Systems Division (formerly the Forecast Systems Laboratory) develops most of the software and modeling applications used by the National Weather Service, NCEP, and other weather agencies (including LAPS).

Remote Sensing and Image Processing Software

• SeaDAS - Image analysis package for SeaWIFS data.
• OSSIM - Open Source Software Image Map project.

See my personal software page for info on some Gentoo ebuilds I currently maintain, as well as some old RedHat/fedora/cAos rpm packages (and the downloads menu above).