swath2grid

Recently I needed to convert swath data to gridded data. Most MODIS products come as gridded products which are properly geo-referenced / rectified. However, some low level products are provided as “swath” data which are the “raw” form when it comes to geo-referencing. Luckily most of these swath products do provide ground control point information to convert them from wobbly sensor output to a gridded geo-referenced image.

This procedure of converting from swath to gridded data is normally done with the MODIS Resampling Tool (MRT) software. Here, I provide a few lines of code which will do just the same using the community driven Geospatial Data Abstraction Library (GDAL). I would argue that the four lines of true code beat installing the MRT tool any day.

The code is a mashup of a stackexchange post and converts MODIS L1B data (or similar) to gridded data, requiring you to specify the file name and the requested scientific data subset (SDS). You can find the available SDS using the gdalinfo command, or using the product information sheet. The data is output as a geotiff.

#!/bin/bash
#
# swath to grid conversion for
# MOD04 data, but will work on other
# MODIS L1 / L2 data as well (I think)

# get the reprojection information, stuff it into
# a virtual file (.vrt)
gdal_translate -of VRT HDF4_EOS:EOS_SWATH:"$1":mod04:$2 modis.vrt

# delete the bad ground control points
sed '/X=\"-9.990000000000E+02\"/d' modis.vrt

# grab the filename without an extension
filename=$(basename "$1")
filename="${filename%.*}"

# reproject the data using the cleaned up VRT file
gdalwarp -overwrite -of GTIFF -tps modis.vrt $filename_$2.tif

 

about complacency on politics

After past week’s election of Donald Trump I’ve decided to break what I consider an unspoken rule in science. Mainly, as a young scientist one does not discuss politics openly, nor take a hard line on issues, as it potentially jeopardizes your academic career.

However, the US elections will have dire consequences for both domestic and global environmental policy and science in general, due to the proposed dismantling of the EPA, overt climate change denial by the president-elect and his anti-science stance. Surprisingly, during the past week the language with regards to Mr. Trump’s victory moved to one of reconciliation, to “acceptance”, giving the Mr. Trump “a chance”.

The ramifications of an unchallenged president-elect and reconciliation will reverberate throughout the globe. Hence, as an ecologist, environmentalist and climate scientist I can not in good conscious stand by, and not take a very strong political but scientifically backed position.

Science and scientists have the obligation to challenge old, and dangerous, ideas. I admit that I’ve failed by not doing as much as I could outside the academic sphere or various social media echo chambers. In this spirit, I will contest misinformation and lies about climate change and environmental issues. As climate change has morphed into an inherently social problem it is also my duty to openly support minorities, poor and the disenfranchised who are the first to suffer from climate change.

I will not concede by giving a man who has shown a lack of scientific knowledge, integrity, transparency and fueled by misogyny and racism “a chance”. I will not lower my voice in years to come, silently accepting the new status quo.

virtualforest.io

Over the summer I created the Virtual Forest project. The aim of the Virtual Forest project is to immersively track the seasons using 360 photography and time lapse movies (once enough data is collected) of a forest in the North Eastern US.

You can track the season in a regular browser, or on a mobile device supporting gyroscope feedback. For a complete immersive experience Virtual Forest supports Google Cardboard and Samsung GearVR. Additional support for VR headsets such as the HTC Vive and Oculus Rift is provided through the A-frame framework.

The project gives everyone the experience of tracking the seasons in a North Eastern deciduous forest through tele-presence. It allows for leaf peeping from a distance and visualizes, otherwise slow, ecological processes using timelapse movies. The images gathered will also support research efforts in understanding the timing of changes to the canopy structure. Above all, I hope Virtual Forest will inspire people to venture outdoors and explore a forest in real life as Virtual Forest remains a proxy for what is a wonderful world.

UPDATE:

virtualforest.io got a nice feature on MotherBoard!

UPDATE 2:

virtualforest.io went viral and hit > 18 000 users in 3 days (and still rising)

UPDATE 3:

The project was featured on HackaDay and the A-frame Blog.

UPDATE 4:

The project was mentioned on the Adafruit Blog.

Parameter estimation in R: a simple stomatal conductance model example

Most modeling approaches use some sort of optimization method to estimate parameters. These parameters are the knobs one has to turn to tune a statistical or mechanistic model exactly right to make it fit the data. Below I give a quick example of parameter estimation (in R) using a stomatal conductance (gs) model.

A collaborator in the lab came to me with the question to quickly introduce him to parameter estimation in R in order to optimize a stomatal conductance model. This is the document and code I used to briefly explain the methodology.

In this example a model calculates stomatal conductance based upon environmental variables and measurements of gmax (or the maximum diffuse stomatal conductance). This model illustrates nicely how one can use R and an optimizer to estimate complex model parameters of non-linear models.

The framework I put together estimates parameters using a Generalized Simulated Annealing optimization R package. However, different optimization packages and methods exist. Some worth mentioning are the default ‘optim’ function from the ‘stats’ package and DiffeRential Evolution Adaptive Metropolis (DREAM). However, I found for smaller projects and quick model development GenSA works really well.

The crude stomatal conductance model as formulated borrows heavily from model structures as described by Jarvis (1976) and White et al. 1999, both succintly described in Damour et al. (2010). The latter paper describes a variety of stomatal conductance models, which could be used with the framework as outlined in this example.

In this model example stomatal conductance is modelled as:

gs = gmax * f(CO2) * f(VPD) * f(PAR)

Here CO2 and VPD (vapour pressure deficit) response curves are modelled using an epxonential equation, while PAR (photosynthetically active radiation) is modelled using Michaelis-Menten kinetics and gmax is measured and calculated as defined by Franks & Beerling (2009).

  • f(CO2) = c1 * e -c2[CO2]

  • f(VPD) = v1 * e -v2[VPD]

  • f(PAR) = ( 2000 * PAR) / (p1 + PAR)

Or condensed and written into an R function this gives:

gs.model = function(par = par, data = data) {

 # put variables in readable format
 vpd <- data$vpd
 co2 <- data$co2
 par_val <- data$par
 gmax <- data$gmax

 # unfold parameters
 # for clarity
 c1 <- par[1]
 c2 <- par[2]
 v2 <- par[3]
 p1 <- par[4]

 # model formulation
 gs <- gmax * c1 * exp(c2 * co2) * exp(v2 * vpd) * (( 2000 * par_val) / (p1 + par_val))

 # return stomatal conductance
 return(gs)
}

Here, the ‘data’ and ‘par’ variables are a data frame and vector including the model drivers and parameters respectively.

The optimization minimizes a cost function which in this example is defined as the root mean squared error (RMSE) between the observed and the predicted values.

# run model and compare to true values
# returns the RMSE
cost.function <- function(data, par) {
 obs <- as.vector(data$COND)
 pred <- as.vector(gs.model(par = par, data = data))
 s < (obs - pred)^2
 RMSE <- sqrt(mean(s, na.rm = TRUE))
 return(RMSE)
}

In the final optimization will iteratively step through the parameter space, running the cost function (which in terms calls the main model), and find an optimal solution to the problem (i.e. minimize the RMSE). Often starting model parameters and limits to the parameter space are required. Defining the limits of your parameter space well can significantly reduce the time needed to converge upon a solution. The main reason is that the model structure on it’s own does not have any sense of the physical reality of the world. If measured values can never be lower than 0 it does not make sense to look for a multiplicative parameter in the negative range of the parameter space.

# starting model parameters
par = c(0.65, -0.002, -0.689, 0.05)

# limits to the parameter space
lower <- c(0,-10,-100,0)
upper <- c(100,0,0,100)

# optimize the model parameters
optim.par = GenSA(
 par = par,
 fn = cost.function,
 data = data,
 lower = lower,
 upper = upper,
 control=list(temperature = 4000)
)

As the data I used in the development of this model aren’t published yet I can’t include any graphs. However, I think that this description can give people a quick introduction in getting started with model development (in R).

References

Damour G., Simonneau T., Cochard H., Urban L. (2010) An overview of models of stomatal conductance at the leaf level. Plant Cell & Environment 2010 33, 1419-38.

Jarvis P.G. (1976) The interpretation of the variations in leaf water potential and stomatal conductance found in canopies in the field. Philosophical Transactions of the Royal Society of London. Series B 273, 593–610.

White D.A., Beadle C., Sands P.J., Worledge D. & Honeysett J.L.(1999) Quantifying the effect of cumulative water stress on sto- matal conductance of Eucalyptus globulus and Eucalyptus nitens: a phenomenological approach. Australian Journal of Plant Physiology 26, 17–27.

Franks, P. J. & Beerling, D. J. (2009) Maximum leaf conductance driven by CO2 effects on stomatal size and density over geologic time. Proceedings of the National Academy of Sciences 106, 10343–10347.

Pi power-over-ethernet

For a project of mine I needed power-over-ethernet (PoE) to be available on my Raspberry Pi. There are several reasons why one would want to use PoE. In my case I need a reliable network connection, and power couldn’t be provided consistently using solar or stand alone methods either. PoE combines both things without the necessity of having to run an extra power cable, proving a rather neat package.

Sadly, the PoE options out there weren’t flexible enough to my liking. Take for example this $40 raspberry pi PoE HAT. Although it provides power to pi there is no option to hook up additional power hungry USB devices. Some finagling would do the trick. However, if I should thinker anyway I better make something that fits my needs.

So instead I made my own little (passive) PoE power solution. A first proof of concept looked like the image above. I used a passive PoE splitter to split the ethernet and the power on the left side (black dongle), to power a $20 uBEC which converts 12-60V into 5V/3A (red shield, I discarded the housing). The 5V output of the uBEC gets connected to the power leads two micro-USB cables (black cables) where one has its TX/RX lines patched through to a  male USB type-A cable. The latter setup allows for data transfer between a high powered device (hard drive) and the Raspberry Pi. This setup bypasses the issue of the PoE hat, which only powers the Pi and consequently can only provide limited power to connected USB device.

Unhappy with this rather ugly solution I remade the same setup using screw terminals and proper type-A port on a larger prototyping shield. The whole setup remains the same but this board now neatly stacks on top of the Pi (with some additional support from standoffs and female header rows). On the left you see a version with only one powered USB port, which powers the Raspberry Pi, and an additional screw terminal output. On the right you see the exact same setup as above, where the left hand USB ports (2x) provide power only, while the right two ports consist of one patch port (connecting a power free data connection) and injecting power and data into the other.

Technically, I could connect power directly to the 5V rail on the Pi, but since I’m not sure how stable and clean the power of the uBEC is I avoid this for now. Power spikes can easily kill GPIO pins or completely fry my Pi. A future iteration might include basic fuses and overpower protection as outlined above. This would limit damage by a less than ideal or reverse input voltages.

But, for now my Raspberry Pi PoE issues are solved.

 

Pagination


© 2018. All rights reserved.

Powered by Hydejack v7.5.1