Time Variation

So I've finally gotten some reasonable numbers for apparent resistivity. It still needs some averaging an I haven't found a good way to achieve that in Matlab (a for loop is too slow because our data set it too large). In the interest of progress, I decided to move forward and come back to the averaging later. I have pulled a 2 hour time period 2 days a month for a year so that I can look at how the resistivity is changing over time (our original goal). I have successfully written a unix shell script to pull the data (since this would take over 100 command lines to do by hand), as well as a Matlab script that will calculate the resistivity of all those data sets and then plot them in a 3d graph vs. time and frequency. Unfortunately, after my first calculations I discovered that some of my data is bad (ie: the instrument was dead or there's weird noise likely due to water in the instrument). This means my next step is to go back and look over the time series of the raw data and throw out the bad data and, hopefully, find new good data to replace it. The good news in all of this is that it looks like I'll actually have results of some sort to present at AGU. Another update coming next week (hopefully with a graph).

So I'm not sure what

So I'm not sure what averaging you need to do, but I have a few possible suggestions. A running average or recursive mean routine could be fast, a decimation would reduce the number of points unless you're concerned with high frequencies, or write a simple compiled program in FORTRAN or C/C++. However, those speedups will only be big if the file input/output is small because that is what takes most of the time in compiled programs and scripts such as matlab or awk have similar problems. Sorry about the bad data, hope you find some good windows and I'm looking forward to seeing some EM results!

That is indeed good news!

That is indeed good news! Perhpas someone here will have some ideas on doing the averaging that you need. Going back through the data will of course be tedious but it sounds like it will greatly improve your results... which will make it worth it. How is all else going? Are you working in the lab or from home?