Random header image... Refresh for more!

Posts from — March 2014

Winston High CPU Usage

As part of my home seismometer network, I set up an installation of Winston Wave Server. It was far easier to set up and get my data (from homemade seismometers) into than Earthworm. However, I noticed that it was running on the hot side. Even when idle, it was using 99% CPU. I thought that was strange, but it wasn’t causing any problems, so I didn’t feel like trying to fix it.

Until today, when I expanded my network to a second node and found that it couldn’t keep up with the data.

Now, I know that people run Winston with a large number of stations and channels. It seemed very odd that two stations and seven total channels was making Winston chug along for me. Sure, I have it on a box that’s not super high-spec, but it should be enough to handle more than seven channels. The docs even say something about running 200-300 stations before needing to optimize.

So clearly, something was wrong.

I reviewed all the config settings. Nothing.

Then I found a “no input” flag, -i. I was running Winston as a service on Linux, so it didn’t need direct input, so I decided to give that flag a try.

Instantly, CPU went from maxed out to barely doing anything.

Not quite sure what the interactive input mode was trying to do when there wasn’t any input for it, but clearly, it wasn’t anything good. Now my server is happily handling seven channels without a complaint, and the server no longer feels like it’s going to burst into flames.

March 23, 2014   No Comments

heli_ewII “No Menu! Just Quit.”

I’m in the process of setting up a seismometer and Winston and Earthworm for a personal seismometer station.  I’m trying to get everything set up to run on startup, so that the system can reboot and continue, without manual intervention.

I’ve got Winston working.

I’ve got my data acquisition script working.

I even have Earthworm working.

The problem is that heli_ewII (which is the only reason I have earthwrom running right now) refuses to play along.  It’ll say it’s starting, then throw a temper tantrum, scream “No Menu!  Just quit.”, and exit.  Whatever it’s doing, statmgr isn’t happy about it, and refuses to retry starting it.  It’ll say that the module needs human intervention.  Later, when I manually restart heli_ewII, everything works fine.  It looks like what’s happening is that the heli_ewII module starts too quickly and is impatient.  I don’t think Winston has had a change to start up and begin serving the menu correctly.  So, heli_ewII thinks it’s talking to a bad server, and instead of gracefully trying again or terminating in a way that statmgr will restart it 30 seconds later, it just explodes.

Clearly not what I want.

Fortunately, Earthworm is open source!  So, why not just fix the code?  This is what I did in Earthworm 7.7.  It’s untested and should probably make the options configurable and might, in fact, do very very bad things.  But hey, it’s free!

Between the lines “Build_Menu (&BStruct);” and “if(!BStruct.got_a_menu) {“, just after the “/* See what our wave servers have to offer */” comment, insert this code.  It should make heli_ewII retry several times before giving up.  Hopefully, this is enough time for Winston to get its act together.

int noMenuCount = 0;
while(noMenuCount < 10 && !BStruct.got_a_menu){
  logit("e", "%s No Menu!  Try again.\n", whoami);
  for(j=0; j<BStruct.nServer; j++)
  {
    wsKillMenu(&(BStruct.menu_queue[j]));
  }
  sleep_ew(5000);
  noMenuCount++;
  Build_Menu (&BStruct);
}

March 9, 2014   No Comments

Earthworm, Winston, heli_ewII, and partial traces

I’m not a geologist.  I’m not a seismologist.  I say that, because I want to make it clear that I have no idea what I’m talking about before you continue reading this.  I want to write about it here because it frustrated me for a couple of days, because there was nothing out there about this issue.

Anyway, I’ve been putting together a personal seismograph station for my house, because I’m just that kind of nerd.  I built a TC1 slinky seismometer last year, and have been running it using Amaseis for a couple of months now.  That software is a bit limited, and I wanted to take it to the next level.  I got an Arduino, an accelerometer, and a Raspberry Pi, and decided that I’d try installing Earthworm.

Because, clearly, an amateur seismometer station hanging out in a garage needs to be running Earthworm.

(I’ll probably talk about my setup at some point in the future, especially if I get it tuned and happy and to the point where I feel comfortable that it’s working and stable.)

Anyway, after wasting several days trying to figure out how to get my data into Earthworm wave_serverV, I gave up and installed Winston, instead.

If it’s good enough for the USGS, it’s good enough for my garage.

I was easily able to write an adapter that fed data from my Arduino into Winston.  Problem is, I don’t really like Winston’s graphs.  They’re so blue.  I like the multicolored heli_ewII graphs more.

So, I pointed heli_ewII at Winston (Yay for interoperability) and didn’t get what I was expecting.  Instead of solid traces, I just got short specks of a trace every two minutes.  Two minutes was the interval to redraw the helicorder images in heli_ewII.d, and it looked like it was only getting a few seconds, then nothing.  I tweaked every setting, turned on debugging, but nothing worked.  I knew the data was in Winston, I could see it in the DB.  But, for some reason, heli_ewII was producing spotty, broken graphs.

Eventually, I sorted out what the problem was.  The data acquisition Python script was trying to be too smart.  You see, Winston wants the sample rate of the data when you insert tracebufs.  Since my Arduino & Pi & Python package was built to the exacting specifications of a project assembled for fun on a kitchen table, the actual sample rate could vary wildly.  However, it was very easy to calculate what the rate was:  Number of samples taken / time taken.  I was sending data to Winston every five seconds or so.  I’d get 280 – 290 samples from the Arduino accelerometer in that window, for a sample rate of about 56-58.  And so, I’d tell Winston the exact sample rate when I sent it data.

That turned out to be a mistake.

I think that variable sample rate ended up confusing heli_ewII and Swarm’s zoomed trace (and spectrograms).  Every time the sample rate changed, they’d stop drawing data.  Swarm’s main heli view worked just fine, but that was it.  Everything else was messed up.

So, I tried giving it a constant sample rate.  That make the helicorder displays happy and continuous, but then I started getting overlapped buffers, or something like that.

In the end, I set my acquisition script to automatically adjust the sample rate it’s sending about once an hour.  This should minimize the missing data in the traces, minimize overlapping data, and still have a sample rate that corresponds to the performance of the accelerometer.

March 8, 2014   No Comments

I guess I never mentioned…

I probably should have said something here, but I promptly abandoned CWP #5 after I discovered the motors I was using wouldn’t propel the ball.  Instead, they just kinda made the motors jam.  I tried adding mass to the motors to give them more inertia, but that just made them off-balance and they started shaking like crazy and there was a burning electronics smell.

Since I clearly had no idea what I was doing and no equipment to fix things, I stopped working on the project.

March 8, 2014   No Comments