News Archive
2012
2012-01-16
Happy New Year! Here is a really nice awk one-liner for getting the minimum and maximum values for the first two columns of a text file:
>>> awk '{if(latmin==""){latmin=latmax=$1}; »
if($1>latmax) {latmax=$1}; »
if($1<latmin) {latmin=$1}} »
{if(lonmin==""){lonmin=lonmax=$2}; »
if($2>lonmax) {lonmax=$2}; »
if($2<lonmin) {lonmin=$2}} »
END {print "LAT RANGE: ",latmin,":",latmax," LON RANGE:",lonmin,":",lonmax}' relocated_simple.txt
LAT RANGE: 30.0310 : 37.5000 LON RANGE: -122.0000 : -113.0832
This is really useful when parsing a >500,000 line text file of earthquake relocations and you need to know the latitude and longitude range for plotting with
a mapping application like GMT or Google Earth.
Adapted from source: http://blog.damiles.com/2008/10/awk-calculate-mean-min-and-max/
(Note: » means that you should join the lines — line breaks added for readability)
2011
2011-12-01
Just in time for the AGU Fall Meeting 2011, here is a sweet shell script that will install 64-bit builds of Numpy (2.0) and Scipy (0.11), Matplotlib (1.2), iPython (0.12), Pandas (0.6), as well as PyMC (2.2 alpha) for OS X 10.7 (Lion) on Intel Macintosh. This should simplify installs for those folks using our new moment tensor code, which we will be presenting the latest iteration of on Friday, December 9th at 8:00am at the Moscone Center.
2011-11-23
Happy Thanksgiving! Here is a cool tool for exploring high-resolution photos.
2011-09-07
The MSNBC-owned website Future of Technology listed Humankind's most ambitious science projects. The project I work on, EarthScope, made the number 1 spot!
2011-09-07
Simple jQuery best practices website with great optimization chapter (9)
2011-07-19
A new tutorial on coding with Antelope and Python - some caveats and gotchas for coding with both
2011-06-07
Here is a great model for using Git in development, which utilizes git-flow, a series of Git extensions to make following the model easy. This how-to explains how to tie all these components together in a simple development model.
2011-06-03
Django dot lookups are pretty sweet, especially when nesting them. I like how they go in the following order of lookup:
dictionary » attribute » method call » list-index. This allows
cool nesting, such as (culled from the Django docs):
>>> from django.template import Template, Context
>>> person = {'name': 'Sally', 'age': '43'}
>>> t = Template('{{ person.name.upper }} is {{ person.age }} years old.')
>>> c = Context({'person': person})
>>> t.render(c)
u'SALLY is 43 years old.'
2011-05-25
I had an intense introduction to the Python yield function, the iterable type
and generators today.
Here
is a great explanation on Stack Overflow about all three.
2011-05-23
Templating rules (ahem, pun intended). Currently comparing Mustache and Jinja. Any other favorites I am missing?
2011-05-20
Today at the monthly Webheads user-group I gave a talk on mobile application frameworks and why I chose jQueryMobile.
Download PDF version (2.9MB)
2011-05-11
Just registered for the Mobile App Hackathon on June 4, 2011:
http://t.co/FRVjFuG
2011-05-11
Playing around with Google web fonts. Make the web beautiful!:
This is some text written using the Lobster font!
Ubuntu is a sweet OS! Just like this font...
Who knew that Allerta Stencil could work on the web?
I think, therefore I am (when I use the Philosopher font)
2011-05-07
There are some really cool Javascript 3D rendering libraries out there these days. three.js is one of my favorites. Check out the demos - they are very impressive.
2011-05-03
Here is a nice ArsTechnica article on Python's role in gaming. A good quote by HPU4Science cluster architect and administrator Yann Le Du:
I enjoy programming, and have used many languages from Fortran to Haskell, through Qi, Ocaml, q, C, Factor, Scheme, Lush, and others. I have some aesthetic feeling for languages, and I must say Python impresses me. It's the perfect choice for someone who wants to write nice, compact code, yet be able to explore wildly beforehand and make good use of specialized libraries. You want to explore? Use Ipython -pylab -p scipy. You want to code some Monte-Carlo integration one-liner? Use built-in Python list-comprehension and some Numeric Python high level functions. Python, for someone with my wide-ranging taste for beautiful languages and need for some practical, powerful language, is a winning card. Really amazing.
Mentioning list comprehensions makes me happy. They are very powerful
and clean (in a code sense).
List comprehensions »
lambda functions » map/filter » for-in loops!
2011-04-28
The hits keep on coming. Now learning about MapBox's TileMill and Carto, a cartography stylesheet rendering language. Dang.
2011-04-28
It appears as though I have turned up late to several cool parties. My friend Marc told me all about Node.js over the weekend. On early inspection it seems like a Javascript version of Twisted's event-driven model. Then my friend Alex told me all about Backbone.js! And then I saw the CSS3 Patterns Gallery! Quoting Ferris:
Life moves pretty fast. If you don't stop and look around once in a while, you could miss it.
2011-01-27
Very tasty screen scraping library for Python called 'Beautiful Soup'. Screen scraping is dirty, but sometimes you don't have a choice.
2010
2010-11-30
NEW Antelope moment tensor code how-to! Want moment tensors in your real-time monitoring? We got it!!
2010-04-04
7.2Mw (6.9ml) event just over the US-Mexico border near Calexico/Mexicali. We recorded the event on our
ANZA network
and USArray network.
In the middle of a media frenzy in my office.... here is clip of our office on the news in Chicago
2010-03-30
Just before April Fool's Day, we finally finished a (almost!) complete rewrite of the Python Twisted
dbwfserver application. This version has full URL parsing, dynamic waveform reloading
on color-scheme change, filter selection, phase arrivals on|off, and waveform|data coverage
bars (to illustrate data gaps in our real time databases. Plus it supports multiple station(s) and
channel(s) selection! The changes have all been pushed to
Github for everyone who is interested.
2009
2009-10-13
Since we have been using Git and Github for our code repository
I thought it relevant to write a how-to on using both these
resources with respect to the Antelope contributed code. Many thanks to Geoff Davis for help
with this!!
2009-09-14
We heard today that for the month of August 2009 the real time data
return rate from the USArray at the ANF was
99.31%! Wow. That trumps the 85% minimum required by
the project, AND data return rates from
any other real time seismic network, globally...
2009-09-09
A little tutorial on
using Antelope's amakelocal to ensure that Antelope
knows about your PHP and Python (and Matlab, etc, ad nauseam)
installations.
2009-07-24
I have been working with Antelope and Matlab a lot lately.
Here is a little how-to on transforming
epoch times that Antelope returns to you and converting them to Matlab-friendly serial-numbers
that can be used to make nice time-axis ticks on your graphs.
2009-05-26
At the Google I/O conference in San Francisco learning about all the
cool new toys Google has been cooking up in its labs...
2009-05-01
Behind the curve on this one.... Just installed
TextMate and added the Git bundle
for revision control. Sweet! And to think I thought Vim was nice....
Here is a good intro on how to use Git to maintain a website:
dmiessler.com
2009-04-22
Spending the day with Google engineers talking about
Google Earth. Learning lots about rendering isosurfaces
at depth, including 3D models generated with
Collada.
2009-04-15
I had to post this, if only for my own information. Some great simple tips for using jQuery efficiently:
10 useful jQuery authoring tips.
Also, some great tutorials by the jQuery plugin guru, Karl Swedberg (who's cluetip plugin I use on this page):
Working with Events, part 1 (Event delegation) and
Working with Events, part 2 (Rebinding event handlers).
2009-04-10
Several of us worked hard to create a
special event page
for the recent Salton Sea earthquake swarm
that occurred in March and April here in Southern California. There were over
450 small magnitude earthquakes in the space of two weeks. The Los Angeles Times
picked
up on the page
and we have seen a x10 increase in web traffic to one of our websites. I just added a
Google Earth layer to download
which shows the temporal evolution of the swarm. This really shows the power of visualization with Google Earth.
2009-02-13
We are converting all our large page inline database queries to a new model in order
to speed up page load times. The long page load time is typically due to database open
table statements, joins (inner and outer) between tables, and image creation. The plan
is to cache database queries as JSON objects, and have the inline code read these
streamlined JSON objects. This also allows AJAX-style page updates based on the updated
JSON object. Comments?
2009-02-01
We have pretty much migrated all of our server side code to Python for creation of data objects that the various
tools on the ANF and ANZA websites parse. I ♥ Python! Way faster than anything I have used before....
2009-01-14
I finally found time to finish the Array Network Facility layer for
Google Earth.
A screendump is shown to the right (click the image for a larger
view). Adding <TimeSpan> elements with child <begin> and
<end> elements allows you to observe the station deployment in
4-D (i.e. time). Clicking a station icon displays a balloon with
station metadata, plus a scaled version of the top pick station
photograph taken by our station engineers and stored on Flickr.
This KML file was created from Antelope, with some XSLT massaging...
You can subscribe to the Google Earth layer by selecting Add > Network Link and entering http://anf.ucsd.edu/cachexml/google_earth/anf_stations.kmz in the Link field. You can assign any name you like to it – this will appear in the Places sidebar as a folder.
2008
2008-12-18
Today I presented my poster at the AGU 2008 Fall meeting. Some good feedback from
people using my Antelope-interface software, plus some new requests for tools.
Should be a busy 2009.
2008-10-30
We are currently experiencing another earthquake swarm just south-west of
the Salton Sea here in Southern California. As of 09:00 we have
recorded over 50 events since 20:30 last night.
Click this link to see an image
of the swarm on a screendump of a Google map.
2008-10-28
Today I was asked to become Associate Editor of the
Electronic Seismologist supplement
of the journal Seismological Research
Letters. I accepted and I am excited to see all the cool new
research going on out there that bridges the science and technology gap.
2008-07-29
I experienced the largest earthquake since I moved to San Diego in 2001
today – a magnitude 5.4 that occurred in the Greater Los Angeles
area near Chino Hills. Both the Anza and
USArray networks
recorded the event really clearly. The TV crews came through and did a
pretty good job, thanks to Debi Kilb's excellent communication skills.
2008-07-28
Today we talked with the developers at SQLstream to see if streaming data
from an Antelope Orb or Datascope RDMS into a SQL stream would help
with some of the NFS connectivity issues we have been having recently.
More on this soon.
2008-07-24
Today I gave a talk on the USArray experiment to the engineers from Google Oceans. They listened to
what myself and several other SIO data managers use Google Earth/Oceans
for, and how we can work together to generate some quality toolsets
that are in line with SIO's mission
statement. We also saw some new functionality in the Google
Earth/Oceans applications, but were told not to talk about them as they
are still under development. Needless to say, they have a few tricks up
their sleeves. I gave a demo of my dynamically created KML file that uses
two XML files as it's source - one from webdlmon and
one from the USArray master database, which essentially creates a
Google Earth version of webdlmon that auto-updates every few minutes.
2008-06-02
Using GMT and Antelope I made
a high-resolution desktop (112MB) and
an iPhone-resolution (172KB) movie of
the USArray station deployment from the start of the experiment (April 2004) to
the present (June 2008). Download and enjoy!
2008-04-23
In between watching the UEFA Champions League
semi–finals I wrote a little Perl script that dynamically creates
Antelope parameter files for the utility orbmonrtd.
There was a need for this because as the USArray network migrates across the country
stations come online and go offline pretty frequently as part of normal
deployment operations. To ensure the orbmonrtd displays are kept up to
date with the latest deployed stations, this script reads the dbmaster
deployment table and outputs snet_sta sources according to some simple
rules (including a modulus
operator and dbprocess commands). Pretty nifty!!! Make sure your
contrib source tree is up to date, and type man
orbmonrtd_pf_generator to learn more.
2008-01-15
I am finally seeing the light and am slowly switching my webapps to
using the Yahoo! User Interface Library for HTML and CSS, and jQuery for Javascript and DOM manipulation. This is a nice visual interface to
jQuery. Many thanks to the multiple developers who have been
nudging me this way for the past six months.
2007
2007-11-20
Last night the Earthscope USArray project I have been working on stopped receiving data from
UC Berkeley stations. Their three year contribution of data to the project
halted as part of normal operations. In my bleary-eyed early morning stumble into my office
I saw the lack of data and was jolted awake. Then I remembered the notification email last night.
For posterity, I took some photos of our monitoring interface before I updated the configuration
file. Goodbye BK stations, and thanks for all the fish....
2007-11-07
It appears that Kent Lindquist, Frank Vernon and my paper came out in
Seismic
Research Letters this week. It is pretty much a summary of what I have been working on over the
last three years, although some of it is now out of date as Kent and I continue to develop the interface,
utilizing AJAX and
Comet (or reverse AJAX, pushlets, etc)
technologies for web–based real–time sensor network
monitoring. Anyway, it is nice to see our work finally in print...
2007-09-27
I have been working on extending the Google Maps API for one of my projects. What I wanted
was the ability to plot multiple nodes at a single location if, for example, multiple instruments
were installed at the same geographical location. This is typical for borehole arrays. So,
I wanted clickable markers that would dynamically display child nodes with lines linking to the
original site. I wanted them to plot on a circle around the station, dynamically determining
distance from each other based on the number of nodes.
Read the article
2007-08-27
I spent the whole of last week working at UC Santa Barbara
at the Institute of Crustal Studies.
They have an Antelope real-time system, and wanted some web-based tools developed. I went slightly
further than I have before with Google Maps, implementing some new features, including dynamic refocused
regions of interest and AJAX queries. I also implemented Smoothbox,
a Lightbox clone for MooTools, my favorite Javascript library.
2007-07-05
Ever wanted to display real-time meteorological data on your website? Read the following article
on how to do just that using PHP5's built-in SOAP functions that query NOAA's web services.
Read the article.
2007-06-21
I have written an article on XSL transformations of XML output, whose origin was an Antelope
parameter file. Note that this how-to is relevant to any XML file, not just Antelope pf files.
Read the article.
2007-06-05
Updated version of the RRD status plots page for the ANF project. It now uses AJAX to dynamically load the RRD plots
under the Google map without a page reload, allowing the Google map state to be preserved. I improved the file size and
load time of the .js file by using JSMin - a
Javascript minimizer that removes comments and unnecessary whitespace from a Javascript file.
Data is returned from the AJAX query in JSON format and allows rapid parsing
via the DOM. I will be writing a tutorial on how I created this mashup soon.
Click to view the application.
2007-05-29:
I just returned from the AGU General Assembly in Acapulco, Mexico,
where I presented some of my recent work, especially webdlmon.
This seems to be a very popular tool with the seismic network monitoring folks. I also presented how
I built the Google Maps and RRDTool mash-up
that allows users to dynamically select a station and choose to plot various state-of-health
parameters. This is still being developed into a more rich application, so stay tuned...
2007-04-27:
I have been using various Javascript libraries over the last few months
in order to make a more rich user interface to some of the websites I
develop. I have experimented with Prototype and
Mootools for
over
six months, and have finally settled on Mootools. The Accordion behaviour
of Mootools drives the navigation menu for this site. It is a very easy to
use system, and is ultra lightweight. Also, there are some very cool
extension kits using the framework - one I particularly like is the
Slideshow class.
I am using MooTools pretty extensively on the ANF site,
especially the homepage.
2007-02-19:
Using a combination of Antelope, XSLT, PHP, AJAX, and RRDtool we have
written a web application version of dlmon. Called webdlmon, it is
essentially a web-based version of the datalogger monitor written by
Danny Harvey of BRTT. It works by cracking open datalogger status packets in
the status ORB, writing out an XML file of the packet values using
pf2xml, and reformatting the XML file using the XSLT parser Saxon.
Using the DOM class built-in with PHP5 and some custom Javascript allows a
table of information to be written on the fly. It
integrates with the RRA's for the state-of-health channels to allow
web-based visualization of any channel parameters.
Check out the application.
2006
2006-10-22:
We have finally got our Round Robin Database Archives (RRA's)
up and running for the Transportable Array (TA)
stations deployed as part of the USArray experiment. These archives
allow graphs to plotted in real-time of all state-of-health parameters
such as vault temperature, datalogger voltage and current, data bytes
read and written, and whether the pump is on (draining water from the
vault!). Thanks to Kent and Geoff for compiling RRDtool with
the Antelope version of Perl. View the graphs here, or choose a station from here and see all plots for the last
day by clicking a station symbol and then choosing the 'State-of-health
metrics' tab.
2006-10-17:
We just received notice that our abstract titled "Monitoring the Monitors: Assessing and Visualizing the
State-of-Health of Earthscope's Transportable Array" has been
accepted for the Fall 2006 American Geophysical Union (AGU) meeting, to be held at the
Moscone Center in San Francisco this December.
2006-09-26:
With some friends and neighbors I competed in a 24 hour mountain bike race in the high desert of California. The
Hurkey Creek 24 Hours of Adrenaline took place on September 16-17. We came in sixth place (out of six!)
in our category (5-people co-ed). Over 300 riders. A strange combination of adrenaline, pain, energy drinks/food, and exhaustion.
Photos...
2006-09-14:
Just back from a two week programming stint in Fairbanks, Alaska with
Kent Lindquist.
We made time to visit Denali National Park. The photo below was taken at mile marker 207, just south
of Cantwell, with Mt. McKinley in the background.
As you can tell from the beard, I went slightly feral up there...
2006-06-20:
Using Dan Coulter's PHP-Flickr
class I am able to
retrieve images from the ANF USArray Flickr photo log on the fly. Images
displayed on the station detail page
are the top pick of the images and photos of the sites from the
cardinal directions.
2006-06-11:
Kent Lindquist and I gave a talk on the Antelope-PHP interface to the
Antelope User's Group
Meeting at the IRIS Meeting in Tucson, Arizona. You can download a PDF version of the talk below.
2006-03-20:
A category 5 Tropical Cyclone
(named "Larry") blasted it's way through
Innisfail, in tropical North Queensland, over the weekend. It pretty
much destroyed the town and
surrounding region. This is very upsetting for me personally as I lived
in Townsville, a couple of hours drive to the south, for 4 years, and
have close friends who have family in the region affected. But, seeing
as I am a visualization geek, I of course made a Quicktime movie of the
satellite imagery of the event. You can view it by clicking on the Quicktime movie link
below.
Read about the event.
Credits: NRL Monterey Marine Meteorology Division
2006-02-26:
My PhD advisor Tim Bell
and I have just had a paper published by the
Geological Society of America (GSA) titled
“Appalachian orogenesis: The role of repeated gravitational collapse”.
The article is available online.
2006-01-10:
This month's PHP International Magazine contains an interview with Kent Lindquist and myself about using PHP for creating web-based environmental monitoring applications for the ROADNet and ANF projects. The article is titled "PHP From the Shop Floor" and was written by Elizabeth Naramore.
2004
2004-05-18:
I won the UCSD 2003-2004 Marine Sciences Exemplary Staff Employee of the Year
award this year. The award ceremony was held on the UCSD upper campus
but I was unable to attend as I was meeting my new nephew Samuel in England.
Last modified: 2012-03-09 20:15:19