Trevor O

ActiveMQ is an enterprise message bus that’s completely open source. It’s great if you want to tie together a bunch of different services, or act as your own personal Simple Message Queue (SQS). It supports a few interface methods such as JMS, Stomp, XMPP and plain REST. You can learn more about ActiveMQ here.

There are a few monitoring solutions for ActiveMQ that will let you know when its broken, but I needed to grab usage patterns over time, so I would be able to automatically spin up more workers. I didn’t see anything quickly available so I threw this together:

#queuemonitor.rb

# queries the activeMQ status XML file and returns relevant data. # I'm sure there's a more elegant way of creating method directly out of xml.   # Rather than have each queuemonitor as a seprate class with one name, we # might want to be able to parse multiple queues on the same server. So we have # an array of queue names (@queues) that contain the names of queues we want to monitor. # if you want to monitor more than one queue then you'd have to create a new QueueMonitor # instance.   class QueueMonitor   attr_accessor :url, :queues   require 'net/http' require 'rexml/document'   def initialize @url="http://127.0.0.1:8161/admin/xml/queues.jsp" @queues = [] end   #add a queue to the list of queues that we'll pay attention to def addqueue(queue) @queues.push(queue) unless @queues.include?(queue) end   def delqueue(queue) @names.reject!{|q| q == queue} end   def query results = [] date = Time.now xml_data = Net::HTTP.get_response(URI.parse(@url)).body doc = REXML::Document.new(xml_data)   doc.elements.each('queues/queue') do |queue| name = queue.attributes["name"] # only list from Queues listed in the '@names' if @queues.member?(name) then   queue.elements.each('stats') do |ele| size = ele.attributes["size"] consumers = ele.attributes["consumerCount"] enqueue = ele.attributes["enqueueCount"] dequeue = ele.attributes["dequeueCount"]   queue = { 'name' => name, 'size' => size, 'consumers' => consumers, 'enqueue' => enqueue, 'dequeue' => dequeue, 'date' => date }   results < < queue end end end return results end   end

You can then start monitoring your queue servers with something like this:

#!/usr/bin/env ruby   require 'queuemonitor'   mfreq = 30 @monitor = QueueMonitor.new @monitor.url = "http://67.202.41.64:8161/admin/xml/queues.jsp" @monitor.addqueue "pubsub.pings.spider"   #do something interesting like write to a file #screen or database   def query date = Time.now results = @monitor.query results.each do |q| puts "#{date} #{q["name"]} #{q["size"]} #{q["consumers"]}" end end   #You'd want to use daemonize here...   loop do query sleep mfreq end

It works pretty well. Right now I’m just writing the results to a file to parse later, and so you can setup other parts of a program that would automatically spin up new workers based on a set of circumstances. The algorithm for that is the hard part and will depend on a bunch of your own rules.

Resisting the urge to say ‘Dude youre getting a Dell’ but this tiny thing is absolutely *perfect* for most home theaters and IT shops. It’s not quite a thin client, but it has enough functionality packed into a small form factor that you’d be silly not to consider using one.

It has HDMI and DVI output, as well as optical out. It supports the full 1080p resolution, which means hooking one of these up to your PC would imply home theater bliss. I currently use a Mac as my home theater, and while it’s nice, it would be a little better to have something small that can run linux, and support the 1080p resolution (and surround sound). At $599, the price is a little steep, but I haven’t seen an HTPC with HDMI/DVI+Optical output for less.

So, if you’re in the market for an HTPC, check out the Dell. (I never thought I’d say that)

Once and awhile I find myself stuck in a very bad habit. I’ll refresh the page of a website constantly, because I love the look, or I want to feel totally ‘up to speed’ on what’s happening in an industry. These are terrible habits, because they make you way less productive, and most sites are just huge distractions with lame out of date material (Slashdot being the best example).

What I do to get around my refreshaholicism is to add the site to my RSS reader (Google Reader) then if I still find myself visiting a news site more than once a day, I’ll add it to my /etc/hosts file so that the DNS doesn’t resolve properly. This lets you continue to view stories via RSS but removes the eye candy - and more importantly makes you stop wasting time. It breaks your habit.

To add a website to your /etc/hosts on a mac or on linux do the following

[509] trevoro 21:42:49 [ hyperion:~ ]
# sudo vi /etc/hosts
Password:

Add the website on the same line that has the 127.0.0.1 entry

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting. Do not change this entry.
##
127.0.0.1 localhost hyperion
255.255.255.255 broadcasthost
::1 localhost hyperion
fe80::1%lo0 localhost
127.0.0.1 popurls.com www.popurls.com slashdot.org

Save and exit. Now when you try and access slashdot.org, you’ll try hitting your own computer instead. If you want to get complicated you can put up some negative reinforcement page that answers requests for all hosts.

Walmart computing is the new term I’m inventing to describe what happens when the Utility Computing or one aspect of the Cloud computing movement makes it bigtime. The idea is simple enough. As more companies access virtual resources instead of real, physical servers then that will result in a reordering of the infrastructure. That reordering will also imply massive cost changes, like I’ve written about before over at LayerBoom.

Once the big companies that offer these virtual services realize how unimportant seeing the actual server is, they’ll just start buying containers of servers instead, similar to what Microsoft is doing with their new data center in Chicago, and that means that on a per-server basis, you’re going to get a way better price. When you’re buying 1000’s of servers, you just let them fail in place, and replace the whole container after a certain threshold (does anyone have any best-practices for this?)

Big companies have buying power, and when you buy things in such large quantities you can get big discounts. Not only that, but you can also justify customized chipsets when you order a million instead of 10. That’s why I call it Walmart Computing. When you’rea huge company with tons of buying power, you get to set your own price. (Perhaps there are even acquisition possibilities?)

There are a lot of interesting papers and blogs written about this subject. James Hamilton over at Microsoft has an amazing paper entitled ‘Emabarassingly Distributed Cloud Services“, and Rob Snevely of Sun published “Enterprise Data Center Design and Methodology” which I have yet to read but have heard great things. What’s most interesting about these papers is that they prefer the modular distributed approach to designing data centers and services, rather than the large bulky approach we use right now. Personally, I’ve been totally converted to that school of thought. James Hamiltons paper has some great numbers but I can summarize it here. For certain services, it’ll cost you about half as much to build the infrastructure in a distributed fashion, rather than a centralized one. The nice thing is, most applications we use online can be “embarassingly distributed”, but that’s perhaps a blog post for another day.

I’m reluctant to write anything remotely soul-revealing and post it online. Everyone’s always supposed to be happy, all the time and nothing bad ever happens because we’re all just so strong, and the rest of the time there’s LiveJournal. I’m not unhappy, but I’m not happy. I’m just around right now. More importantly I just felt like it was time to write something that didn’t make me sound like a robot. What, with such cerebral titles as “SOHO Cloud Cache”, “Rogers DNS Hijacking”, “Supernerd Geek Moment”, and my personal favourite “Data Transparency and Queries” I’m surprised so many people pay attention.

Circumstances being what they are, I haven’t been comfortable for quite some time. Don’t get me wrong, that’s a good thing. But sometimes I feel like a goddamned mini-wheats commercial (How’s that for bad analogies?). The kid in me loves the adventure of pushing the envelope, and trying new things, while the Adult in me just wants a steady paying job and a place I can call home for a few years. But just like any cereal not covered in sugar, the safe side is usually goddamned boring.

Previous experience has taught me well. Anything worth doing won’t be easy, and easy things are rarely worth doing. School, Travel, Adventure, Relationships. The tough things are the ones that resonate and define you and the easy things? Well they were just distractions. Sometimes things pile up and it’s a little overwhelming. When do you get to just chillthefuckout? What’s the line between foolishness and ambition, and when do you chase after something without question? Do you fight for Love or cherish Love lost? It’s important to know people that make you feel truly comfortable, at ease and happy in each others presence. Attempts to draw on previous experience are proving to be futile.

I’m losing someone important to me soon. The fact that it’s on a calendar doesn’t help I don’t think. Perhaps sudden departure would be better. There’s no bad blood, no fight, no real reason other than an unfortunate set of circumstances, which if they didn’t exist would mean a completely different reality. Part of the problem is probably because I ask too many questions, and postulate too many possibilities - ‘What ifs?’

What exercise or elixir shall ease this relentless emotional curiosity?

Hi everyone!

I’m looking for some help answering some questions to help my company LayerBoom and some associated projects determine what companies and individuals really want in their own “Cloud Computing” environment.

I know the Term has been getting worn out, so to summarize, LayerBoom is providing software that helps companies build utility computing environments using existing hardware and network infrastructure. The idea is to provide software as a common platform, then provision resources across many distributed environments, and enable companies and academic environments to rent/share unused resources for extra revenue.

There’s a survey posted up at http://survey.layerboom.com

I’d appreciate your help and time!

-Trevor

Gary Vaynerchuk of WineLibrary.TV spoke at Web2.0 expo and manages to give what I consider the best piece of advice that too few people hear. Too many people work jobs they hate so they can buy stuff they don’t need, all the while longing to do something else.

The short and sweet of it: Do what you love, stop bitching about hard work, and find the time to build things you care about.

Here’s an email I got from the Minister of Treasury. It sounds like they really need our help. I would encourage every one of you to forward this message, unsolicited to your friends, family and colleagues.

I need to ask you to support an urgent secret business relationship with a transfer of funds of great magnitude.

I am Ministry of the Treasury of the Republic of America. My country has had crisis that has caused the need for large transfer of funds of 800 billion dollars US. If you would assist me in this transfer, it would be most profitable to you.

I am working with Mr. Phil Gram, lobbyist for UBS, who will be my replacement as Ministry of the Treasury in January. As a Senator, you may know him as the leader of the American banking deregulation movement in the 1990s. This transactin is 100% safe.

This is a matter of great urgency. We need a blank check. We need the funds as quickly as possible. We cannot directly transfer these funds in the names of our close friends because we are constantly under surveillance. My family lawyer advised me that I should look for a reliable and trustworthy person who will act as a next of kin so the funds can be transferred.

Please reply with all of your bank account, IRA and college fund account numbers and those of your children and grandchildren to wallstreetbailout@treasury.gov so that we may transfer your commission for this transaction. After I receive that information, I will respond with detailed information about safeguards that will be used to protect the funds.

Yours Faithfully Minister of Treasury Paulson

Pirate day makes me nostalgic for a time when every day was pirate day. I spent some time sailing on the Soren Larsen out of Auckland, NZ. Sailing is easily the best way to travel the world. Remember - Its the journey, not the destination. :p

Yarrr I never cut me hair or shave.

Good times.

There have been a lot of startups entering the cloud storage space lately, and quite frankly not too many of them are doing anything interesting. Most business plans revolve around backing up data in S3, building a nice client and doing some version control on files. I know how difficult it is to do that well, but something that can and has been repeated over and over is going to have to really blow the doors off to make any kind of impact - the margins are too thin. There are exceptions. I really like JungleDIsk, and the way its setup. I’m also appreciative of Mozy and their ability to get acquired by one of the larger companies who like to get their chequebook out. Om Malik has written about it and hes not impressed, and I appreciate it when respected peoples opinions mirror my own.

So does anyone in the storage space want to make some extra cash? Here’s a free idea.

All those SOHO storage platforms that exist out there, like Drobo and Buffalo - Integrate some cloud computing backup into your system. It goes a little something like this.

I have a terabyte of storage locally for all my music, movies and some other important documents, and its mounted from my computer over the network using regular AFS or CIFS sharing. All you do is setup the storage systems default layout so that it has a ‘protected’ folder. The protected folder will automatically sync and version all the files to the storage cloud of your choice (probably S3 or Nirvanix). Of course things get tricky as you wouldn’t want to drop in a 4gig movie, and have that eat up your network - so there would have to be format and size filters to setup as well. Depending on the % of free space you can also move unused files into the cloud only, but still display their availability locally.

There are a lot of companies and individuals that have these systems in their office for sharing files between users, and the more you synchronize, the more they effectively become SOHO caches.

Update: Jim Pick has told me there’s a company named Wua.la that does something similar to this. Wuala’s model seems more like a P2P distributed storage system - Similar to AllMyData’s Tahoe storage system.