Tuesday, November 03, 2015

Evolution of the waveform suite

The History of the WAVEFORM SUITE for MATLAB

While in graduate school, I wrote this MATLAB toolbox called "The Waveform Suite" (2011 paper by Reyes & West) to solve a problem that I was having.  I was working with seismic data retrieved from a proprietary database, on a system that barely functioned, with some codes that were hand-me-downs.  I needed to take the data, run a bunch of metrics on it, and then store the results.  Furthermore, I wanted to be able to work on my laptop or my home computer and not be limited to the archaic SUN machine that was my university workstation.  

First iteration - The messy desktop

It became quickly apparent that I spent an inordinate amount of time trying to wrestle with the data and metadata, which lay in a pile of variables on my MATLAB workspace.  Something like 
Sta  Cha  Sta2 starttime1 freq1 freq2 starttime2
Cha2  D  DD  DDfilt1 npoles1 npoles2 nyq 
lat1 lat2  lon1  lon2 filttype1 filttype2
wSize nfft minfiltfreq maxfiltfreq ans n m idx cntr

You get the idea.  You had to rememver that lat1 & lat2 belonged to Sta2, which had data DD, but that you filtered into a variable called DDfilt1.  Yes, some of the variable naming conventions were my own fault.  But even when the variables were created with easy-to-follow names, you still had to chase them everywhere.   The result was heavily dependent upon direct input from the user, one command at a time.  These operated with a few scripts that would:
  • get data
  • filter data
  • display spectrogram
to change which data you retrieved, or which filter was applied, or the display aspects of the spectrogram, you needed to edit a parameter file.  Otherwise, everything occurred in the main workspace. 

Each script worked with the variables in the main workspace, and were hardwired to specific variable names, either in the script itself or the accompanying parameter file.

Second iteration - Functions and Structs

The variables were tamed by grouping related values into structs, and the pile of global variables was tamed by turning the scripts into functions.  These two simple steps provided the most "bang for the buck". And I recommend them for every MATLAB program.

From individual variables to structs

For example, the "filter" information could be kept together using a struct with fields.  So, this kind of behavior replaced

npoles1 npoles2 filttype1 filttype2 minfiltfreq maxfiltfreq

filt1.type         filt2.type
     .freqrange         .freqrange
     .poles             .poles

Likewise, I could package the station and channel information together with the samplefrequency, starttime, and data. 

From scripts to functions

And the accompanying filter function could accept data and one of these filter variables. 
function D = filtfilt(myfilt, data, sampfreq);
  % code that filters using myfilt.type, myfilt.freqrange, and myfilt.poles

Now, instead of having to hard-code each operation and variable,  I could pass any variable into the function and get well packaged behavior.  The additional benefit was that my workspace no longer contained all the temporary variables, such as n, m, idx and cntr which were only needed in a loop, or during some intermediate step.

Third Iteration - Classes

At this point, I had already significantly cleaned up my workflow and was fairly pleased with the results.  However, a few annoyances remained.
  1. Difficult to debug run-time errors.  Sometimes unexpected, bad or unassigned values could cause crashes or (worse) incorrect answers.  These bad values, however, might have been assigned minutes ago or even weeks ago.
  2. Working with multiple waveforms required the user to create loops. This added to the number of lines of a code and reduced code readability
  3. User had to specifically calculate ancillary data, such as the vector of sample times, the nyquist frequency, or period.
  4. User would have to always specify the field that they're working on.  like w.data
The solution to #1 was to check values as they were assigned to the struct.  How? By using classes. Classes are super for hiding the ugly internals of the calculation and can intercept assignments, allowing you to do error-checking at the moment a value is assigned.  Better yet, when you create these checks, you can also either compensate for the bad data, or provide a useful (actionable) error message. 

Using classes could also fix #2, because the logic for dealing with multiple waveforms could be hidden, allowing the user to merely specify an operation on an array of waveforms. 

Classes provide the way to associate class-specific functions (AKA methods) with the data in the class.  Therefore, you can create functions to automatically calculate ancillary data, (#3).  An additional benefit is that standard operators (like +, -, /, etc) can be rewritten to specifically handle the details of the class.  For example

At this point, we've taken something like:

for n=1:numel(w)
  w(n).data = w(n).data + 1 % offset all data by 1
  w(n).freq = 20            % set the sample frequency
[w.freq] = deal(20);        % alternate method of setting freq

and allowed it to be rewritten as:

w = w + 1;               % offset all data by 1
freq = set(w,'freq',20); % set sample frequency to 20 for all

The first line  is much easier to read and deals with n-dimensional waveforms.  The second line became a little more convoluted, but provides data validation, and ensures that we can't accidentally assign [20 20] or 'oops' to the frequency values.

Fourth iteration - From old-style classes to the new 

Moving forward, we can take advantage of the way that MATLAB now handles classes to make the previous example even more readable:

w = w + 1;
[w.samplerate] = deal(20);

Although the samplerate usage is the same as it was for the struct, it is now validated!

How classes used to be

Classes in MATLAB were defined as a collection of files, all stored in the same folder named with @ + classname.  example: @waveform, @scnobject, etc.

Values within the Waveform Suite classes were accessed using "get" and "set" functions, much the same way that one uses them with graphics objects:
cv = calcval(get(w,'nyquist'), otherparms); 

How classes are now

cv = calcval([w.nyquist], otherparams);

While the old usage is still valid, the newer way to create classes is to have all the functions in the same file and there is much greater control over the variables that each contains.  They can be accessed as though they were structs, but can have value checking and /or be calculated on the fly.  The changes are pretty sweet and I look forward to continuing to work on them.

You might have noticed that there are square brackets [ ] around w.nyquist.  When I ask for a class property (variable), MATLAB returns each one separately.  The brackets group them together and prevent the "too many input arguments" error.

The Future of the WAVEFORM SUITE for MATLAB

The waveform Suite had been hosted on Google Code, which has closed.  It has now been migrated to GITHUB and is the core component of the GISMO suite, hosted under  "Geoscience Community Codes".  

  I am in the process of rewriting all the core classes in the newer style, and have still got some testing and standardization to do before I release it into the wild.  However, I am excited about the upcoming changes.

Switzerland update

So. Apparently I haven't written in the blog since I first arrived.  I've been busy.  Since I've been in Switzerland I've:
  • Gotten Married (twice, if you count the "official" ceremony and the public celebration separately)
  • Learned a bunch of German.
  • Worked one autumn deep in several valleys scattered throughout the country.
  • Searched for jobs. 
  • Returned to Alaska to finish my PhD. (Yay!)
  • Eaten lots & lots of cheese, sausages, yogurt and Muesli.
  • Rode many trains.
  • Hiked around in the Alps
  • Been to Europa Park (Germany), and The World Fair Expo (Milan, Italy), and Paris.
  • Joined a choir.
  • Started earnestly reworking my MATLAB seismology codes.
I should have written more.  Already, life in Switzerland doesn't seem so strange. 

Wednesday, February 05, 2014

Down die Kaninchenbau - Wilkommen in der Schweiz

The train glides quietly past apple orchards and fields and hamlets.  Along the way, hillsides are dotted with small clusters of houses that spread gently out from church spires. Conversations around me are occurring in Swiss German (Burreduetsch) and Italian; but I sometimes hear French, German (Hochdeutsch), or even English.  The day is beautiful, with a clear sky and snow-capped mountains that dominate the far landscape.

Today is my first full day in Switzerland, and I've almost missed it.  Jet-lag has inspired me to sleep until 11am, when I was roused by a phone being handed to me. Blearily I answer the phone that I'm offered, and get an update from Barbara.  My sister was trying to get a hold of me.  I called her back via skype.

She answered in a voice that sounded more tired than even I was. Oh. it's 4am there.  Apparently, I'm not at my most alert when I'm jet-lagged, and forgot about the 6 hour time difference.

I'm on a train on my way from Buchrain to Zurich, where I"ll meet a couple of friends (Marijke and Brad) for a coffee.  The train is passing through a mixture of old wooden houses, complete with shingled sides and tall roofs, and modern industrial areas.  The combination of Concrete and Wood is quite interesting.  Hillsides are beautifully manicured with small forests punctuating the fields.  Here and there, a small cartoon character is displayed from the top of a pole, announcing the birth of a child (apparently, it's the godfather's responsibility to make these proclamations).

Moments ago, I made the switch at Rotcreuz.  (Red Cross).  When leaving Buchrain for Zurich, there is no direct train, and you'd either need to switch at Rotcreuz (for trains leaving at :32) or Zug (for trains leaving on the hour). As expected, the change was seamless. Trains on this section of track seem to always be on time. And by "on time", I mean "arrive and depart within seconds of the expected time".

Overall this particular ride will take about 50 minutes (I'm too lazy to look up the exact time... but it would be exact).

When I arrived in Zurich yesterday, the weather was schlect. Fog & clouds.  Barbara picked me up at the aiprort at a little before 8.  Passing into this country is a true joy compared with going to the US.  Only one question at immigration ("How long..?") and a direct route from picking up your bags to

[Oh my. I just saw double-decker bike racks.  I was too surprised to take a photo.  They're fair game next time I ride this route.]

... where was I?... direct route from where you grab your bags to the outside, so long as you have nothing to declare.   That's one thing I already love about this country.  They treat you like an adult. Another example: Buy your tickets & board a bus or train.  Nobody looks to see if you've done it [most of the time], it is just assumed that you do.  There will be spot checks, and you're in big trouble if you didn't buy a ticket; but it's not like they automatically assume you're screwing the system. Plus, they don't spend $$$$ trying to make sure that every individual pays. So what if a few bad apples don't pay? The system remains friendly and efficient and free of turnstiles.

[We just passed through a tunnel and are now on the Zurich side of some place or another driving through a pretty little wooded valley with a stream running down the middle.]

Back to the arrival... Barbara picked me up and we drove back to her parent's house. When we arrived, I noticed a beautiful handmade sign they'd set up. It was a cut-out wooden switzerland shape (painted red), with "Wilkommen dahaim, Celso" written in colorful letters; all against a blue background.  What a great welcome!

Welcome home.  Yes, I'm now in Switzerland for the long run.  After much planning and shedding and selling and farewelling, I'm finally here.  This isn't the first time I've been in Switzerland, but it's the first time that it seems permanent. With that thought, I figured I'd better start writing things down.

[right now, we're passing along a long lake, about to arrive at the Thalwil pronnounced something like "Tall-veal" station]

We ate croissants, "Gipfeli" and yogurt "Yogurt" for breakfast, after which I promptly took a long nap.  We woke in time for lunch, and then started running around taking care of the important stuff... like purchasing house slippers. oh, and starting the paperwork necessary for our wedding. ;-)

Barbara's father also made me a most-excellent extension cord that translates the swiss plugs to American-style plugs.  The voltage is still the same, but most adapters these days can run on either 240v/50cycles OR 110/60cycles.  With that, I become electronically invincible.

[I'm in a tunnel again, and am guessing that I'm only about 5 minutes away from arriving in Zurich. So, I'll end this post here.]

Somebody poke me if I start to take a long time in between posts; since I really would like to document my initial impressions of life in Switzerland.  I'm really curious how my impressions will change through time.

Cheers / ciao / Tseuss,

Monday, March 29, 2010

The final meltdown

So-- first thing: The ice park people were thinking about placing our saguaro into the Ice Museum! That's a refrigerated building downtown that's open all summer, and where the tourists can go to get their ice-art fix. Very cool, though it would entail chopping the saguaro up, and rebuilding it in the museum. However, due to severe melting (see next paragraph), this may no longer be an option, though there's still a chance we may rebuild one there.

The ice park's final day was yesterday, and most of the sculptures lasted up until the very end. Well, the last day, anyway. Warm temperatures these last couple days, combined with breezes have chewed up some of the sculptures somethin' fierce. A few had fallen over, while others were highly eroded nubs of their former selves. One, a single block "Cell R' Ice" was standing when we first walked by (before the lights were turned on for the night), and had fallen by the time we made it back around later that evening. Several statues were making faint cracking sounds.

I'll put up a few before/after shots when I get the chance.

Saturday, March 06, 2010

That's a wrap!

Wow, what an amazing week. I can barely type, my wrists and trigger finger are so sore. After 5 1/2 days of redesigning, sawing, grinding, chiseling, prying, dragging, scaffold climbing, gluing, freezing, more grinding, and torching, create "Sonoran Sunset". Now, all we need to do is a little more cleanup and drop off the lighting diagram and we're Done! Done! Done!

I wonder what we'll carve next year....

Thursday, March 04, 2010

Day 4 - Saguaro stands TALL

What a great day! The wall is finally complete, we've lifted our sections (mostly cut by Tim) onto the saguaro, which now stands at approximately 18 feet. Katie came in and whipped out a baby javelina (off camera), the cougar is now taking shape in its' cave, Jill's prickly pear pieces are in place. It's amazing to see it all coming together.

We had a mad rush, and ended at 2am last night, because we needed to get all large pieces glued into place so we could start working them today. Today, all the rest of the elements will probably be in place, so that tomorrow we can spend the entire day going over our sculpture and texturing it.

Right now, we have nearly 2 unused blocks worth of ice... which will go a little towards our river, but will likely remain mostly unused. oh well... the desert doesn't need to be crammed full of stuff, anyway. :)

Wednesday, March 03, 2010

next day....

The saguaro midsection has been carved, but is off camera. Also, some prickly pear. Katie arrived in the evening as we were mounting the cougar cave. Feelin' good! and tired.
Catch the action here, or at icealsaka.com

Tuesday, March 02, 2010

multiblock - second day

A really quick summary: We built wall. We built wall from 10am to 1am. As usual, there were plenty of little hangups, like pieces that don't quite connect. But, the wall is essentially done, and we can finally move on to other stuff. Today: Tim'll probably work on the cougar, Jill will probably work on her prickly pear, and I'll be running around like a chicken with his head cut off.

Monday, March 01, 2010

First day Recap

We started working with our ice as soon as the 9am horn sounded. Jill and Tim started by trimming the orange marker marks (made during the ice harvest) from the edges of the ice, and then cutting the white ice off of a block or two. Meanwhile, I was trying to figure out which ice blocks needed to go where for the wall, and strode around the site with a measuring tape trying to figure what-goes-where.

After leveling the site (a futile effort I was to learn) we pulled our first block up into position. That big vertical block in the back is going to be the base of our cougar, and anchor for the background wall.

Another block we had moved into a vertical position to stage right. That is going to be the base of the Saguaro. Unfortunately, we learned after the first cut, the saguaro block is riddled with fractures. Actually, most of our ice is very fractured. This may be a side-effect of such a low snow year, because the snow, which cushions the ice as the forklifts push, pull, and drop each bock, was AWOL. We got to experience this throughout the day as each of our blocks were in turn abused and fractured. Mimi, a few sites down, had one of her blocks completely crumble.

SOoo, with the ice not behaving ideally, we spent some time wandering around our site trying to decide just how bad the damage was. Our tool: An iron which we used to clarify the blocks enough to look inside. Our answer: pretty bad, but we'll give it a go. If things get really bad, we have the option to get 2 replacement blocks of ice.

One of the challenges we face comes from the depth of the ice. Our ice is somewhat thick this year, and while that's nice, it also means that even our longest chainsaw can't cleanly cut through an entire block in any direction. That's why you'll see us circling the blocks with chainsaws, and cutting in at both sides.

Our next little setback was when we tried to split the third block in two. The resulting pieces were to be stacked upon each other, making up the wall right next to the large vertical block. The back piece fell over and shattered. (another victim of little padding). We had to split the remaining piece to make up for this. We opted, however, to do it in smaller sections. which would also go to make up the wall. As for the shattered bits? They'll make nice craggy bits to top our wall.

All this took quite a while. Meanwhile, off camera, one block had the edges rounded off of it. That block was destined to become a prickly pear bush. Another block was marked to become the middle section for the saguaro.

Somewhat near the end, we had a couple blocks pushed into place and glued.

Lessons learned yesterday:
  • Whenever a block is going to change orientation or has the chance of falling out, scoop lots and lots of snow to cushion it
  • Think ahead of time about what ice is going to fall where, and make sure that nothing else you care about is nearby.
  • Get scaffolding early in the competition. We don't really have any, as it's all doled out to other teams.
  • Move thicker blocks and wrench them into position to carve, rather than trying to move anything delicate.

Today's assault: Finish the wall, get the saguaro to level2, start saguaro #2, and get the prickly pear into position. We may even get to cut out a block specifically for the cougar/cave, and another couple blocks for the javelinas. Of course, this plan is subject to change....

Sunday, February 28, 2010

AAAAaand they're OFF!

We registered yesterday, and will be at site #7. Tim's got gloves, Jill *finally* got her chainsaw, and Katie will be joining us Tuesday night.

Somehow, after lots of planning (and continuing through this writing), we've finally survived to our first day. The blocks look thick, and the site is smaller than anticipated (20'x25'). NOW it starts, and you can expect this blog to get terser and somewhat less coherent as the week wears on.

Yesterday was mostly a break, with a trip to Chena Hot springs with other sculptors before we returned to the ice park for final registration and a safety briefing.

Today, if the camera is up and working, expect to see us clearing the area, scraping orange marker off the ice, and then cutting the large blocks of ice for the saguaro and backdrop. Catch all the action HERE: http://icealaska.com/videocam/10Cam/mb10/CAM10mb07.html
The camera will take a new picture every minute or so, so if we've been stationary for a while, refresh the browser.

Deep breath...