Backup or Store stuff to GMail via IMAP in Ruby

Once upon a time, I used to store some automated small backups into GMail just by having the scheduled backup send an email to my GMail account. At one stage they blocked me from doing so, marking those repeated email as SPAM.

After that, I took a different approach: I kept sending the mail on the SAME server as the backup, and using IMAP I could DRAG-and-DROP the backup attachment from the mailbox on one server to the mailbox on another server (=GMail). They did not mark me as a spammer that way, of course.
So that worked for a while, but then I got tired of doing this manually.

So the following ruby script is the way I automated the “move offsite” part of that backup.
For completeness, I will give the due credits about who set me on the right track: I started off by this example by Ryan.

#!/usr/bin/env ruby
begin_ = Time.now

#includes
require 'net/imap'

##Source Info
$SRCSERVER="mail.muscetta.com"
$SRCPORT=143
$SRCSSL=false
$SRCUSERNAME="daniele"
$SRCPASSWORD=""
$SRCFOLDER="INBOX.Backups"

##Destination Info
$DSTSERVER="imap.gmail.com"
$DSTPORT=993
$DSTSSL=true
$DSTUSERNAME="muscetta@gmail.com"
$DSTPASSWORD=""
$DSTFOLDER="Backup"

#connect to source
puts "connecting to source server #{$SRCSERVER}... nn"
srcimap = Net::IMAP.new($SRCSERVER,$SRCPORT,$SRCSSL)
srcimap.login($SRCUSERNAME, $SRCPASSWORD)
srcimap.select($SRCFOLDER)

#connect to destination
puts "connecting to destination server #{$DSTSERVER}... nn"
dstimap = Net::IMAP.new($DSTSERVER,$DSTPORT,$DSTSSL)
dstimap.login($DSTUSERNAME, $DSTPASSWORD)
dstimap.select($DSTFOLDER)

# Loop through all messages in the source folder.
uids = srcimap.uid_search(['ALL'])
if uids.length > 0
	$count = uids.length
	puts "found #{$count} messages to move... nn"

	srcimap.uid_fetch(uids, ['ENVELOPE']).each do |data|
		mid = data.attr['ENVELOPE'].message_id

		# Download the full message body from the source folder.
		puts "reading message... #{mid}"
		msg = srcimap.uid_fetch(data.attr['UID'], ['RFC822', 'FLAGS', 'INTERNALDATE']).first

		# Append the message to the destination folder, preserving flags and internal timestamp.
		puts "copying message #{mid} to destination..."
		dstimap.append($DSTFOLDER, msg.attr['RFC822'], msg.attr['FLAGS'], msg.attr['INTERNALDATE'])

		#delete the msg
		puts "deleting messsage #{mid}..."
		srcimap.uid_store(data.attr['UID'], '+FLAGS', [:Deleted])
		srcimap.expunge

	end

	#disconnect
	dstimap.close
	srcimap.close
end

total_time = Time.now - begin_
puts "Done. RunTime: #{total_time} sec. nn"

CentOS 5 Management Pack for OpsMgr SCX

As I mentioned here, I have been testing the SCX beta.

Not having one of the “supported” platforms pushed me into playing with the provided Management Packs, and in turn I managed to use the MP for Red Hat Enterprise Linux 5 as a base, and replaced a couple of strings in the discoveries in order to get a working CentOS 5 Management Pack.

CentOS_HealthExplorer01_NEW

I still have not looked into the “hardware” monitors and health model / service model, so those are not currently monitored. But it is a start.

A lot of people have asked me a lot of information and would like to get the file – both in the blog’s comment, on the newsgroup, or via mail. I am sorry, but I cannot provide you with the file, because it has not been throughly tested and might render your systems unstable, and also because there might be licensing and copyright issues that I have not checked within Microsoft.

Keep also in mind that using CentOS as a monitored platform is NOT a SUPPORTED scenario/platform for SCX. I only used it because I did not have a Suse or Redhat handy that day, and because I wanted to understand how the Management Packs using WS-Man worked.

This said, should you wish to try to do the same “MP Hacking” I did,Β  I pretty much explained all you need to know in my previous post and its comments, so that should not be that difficult.

Actually, I still think that the best way to figure out how things are done is by looking at the actual implementation, so I encourage you to look at the management packs and figure out how those work. There are a few mature tools out there that will help you author/edit Management Packs if you don’t want to edit the XML directly: the Authoring Console, and Silect MP Studio Lite, for example. If you want to delve in the XML details, instead, then I suggest you read the Authoring Guide and peek at Steve Wilson’s AuthorMPs.com site.

Disclaimer
The information in this weblog is provided “AS IS” with no warranties, and confers no rights. This weblog does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my own personal opinion. All code samples are provided “AS IS” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.
THIS WORK IS NOT ENDORSED AND NOT EVEN CHECKED, AUTHORIZED, SCRUTINIZED NOR APPROVED BY MY EMPLOYER, AND IT ONLY REPRESENT SOMETHING WHICH I’VE DONE IN MY FREE TIME. NO GUARANTEE WHATSOEVER IS GIVEN ON THIS. THE AUTHOR SHALL NOT BE MADE RESPONSIBLE FOR ANY DAMAGE YOU MIGHT INCUR WHEN USING THIS PROGRAM.

Popfly Virtual Earth Mashup on Moonlight

Popfly Virtual Earth Mashup on Moonlight


Installed moonlight on Ubuntu from source by following these instructions (there are some typo’s but they are understandable and correctable).

All in all, even being still under heavy development, what Miguel de Icaza has achieved (with moonlight, just like with mono) is amazing.

After I posted the above picture on Flickr, John Montgomery was amazed to see PopFly (his creature) working on moonlight, and he linked to me from his blog.

Looking at OpsMgr2007 Alert trend with Command Shell

It’s friday night, I am quite tired and I can’t be asked of writing a long post. But I have not written much all week, not even updated my Twitter, and now I want to finish the week with at least some goodies. So this is the turn of a couple of Powershell commands/snippets/scripts that will count alerts and events generated each day: this information could help you understand the trends of events and alerts over time in a Management Group. It is nothing fancy at all, but they can still be useful to someone out there. In the past (MOM 2005) I used to gather this kind of information with SQL Queries against the operations database. But now, with Powershell, everything is exposed as objects and it is much easier to get information without really getting your hands dirty with the database πŸ™‚

#Number of Alerts per day

$alerttimes = Get-Alert | Select-Object TimeRaised
$array=@()

foreach ($datetime in $alerttimes){
$array += $datetime.timeraised.date
}

$array | Group-Object Date

#Number of Events per day

$eventtimes = Get-Event | Select-Object TimeGenerated
$array=@()

foreach ($datetime in $eventtimes){
$array += $datetime.timegenerated.date
}

$array | Group-Object Date

Beware that these “queries” might take a long time to execute (especially the events one) depending on the amount of data and your retention policy.

This is of course just scratching the surface of the amount of amazing things you can do with Powershell in Operations Manager 2007. For this kind of information you might want to keep an eye on the official “System Center Operations Manager Command Shell” blog: http://blogs.msdn.com/scshell/

Get-FlickrPhotos

A while ago, talking to some friends, I was mentioning how cool it was that Flickr provides APIs, so that you can always get your data out of it, if you want to. There are several downloader applications that I found on the Internet, but I have not yet chosen one that I completey like among the few that I’ve tried. So, inspired by Kosso’s PHP script for enumerating your photos on Flickr, I thought I’d port it to Powershell and make my own version of it. Just for the fun of it. My Powershell script does not do everything that Kosso’s one does: I don’t build a web page showing description and comments. I suppose this is because the original script was made with PHP, which you usually run on a web server and outputting as HTML is the standard thing you would do in PHP. I just concentrated on the “download” thing, since mine it is a console script. You can think of mine as a “full backup” script. Full… well, at least of all your photos, if not of all the metadata. It should be trivial to extend anyway, also considering Powershell XML type accelerator really makes it extremely easy to parse the output of a REST API such as Flickr’s (I would say even easier and more readable that PHP’simplexml). There is a ton of things that could be extended/improved in the script… including supporting proxy servers, accepting more parameters for things that are now hardcoded… and with a million other things. Even this way, though, I think that the script can be useful to show a number of techniques in Powershell. Or just to download your photos πŸ™‚ So you can download the script from here: Get-FlickrPhotos.ps1

Welcome www.powershell.it!

I just read from Jeffrey Snover about this newly born Italian PowerShell community site.

I just created an account for myself on the site… as you know I like PowerShell, so even if I usually prefer writing stuff in english, I will try to hang out there and see how can I contribute to it.

After all, I am italian… πŸ™‚

Simply Works

Simply Works

I don’t know about other people, but I do get a lot to think when the end of the year approaches: all that I’ve done, what I have not yet done, what I would like to do, and so on…
And it is a period when memories surface.

I found the two old CD-ROMs you can see in the picture. And those are memories.
missioncritical software was the company that invented a lot of stuff that became Microsoft’s products: for example ADMT and Operations Manager.

The black CD contains SeNTry, the “enterprise event manager”, what later became Operations Manager.
On the back of the CD, the company motto at the time: “software that works simply and simply works”.
So true. I might digress on this concept, but I won’t do that right now.

I have already explained in my other blog what I do for work. Well, that was a couple of years ago anyway. Several things have changed, and we are moving towards offering services that are more measurable and professional. So, since it happens that in a certain job you need to be an “expert” and “specialize” in order to be “seen” or “noticed”.
You know I don’t really believe in specialization. I have written it all over the place. But you need to make other people happy as well and let them believe what they want, so when you “specialize” they are happier. No, really, it might make a difference in your carrer πŸ™‚

In this regard, I did also mention my “meeting again” with Operations Manager.
That’s where Operations manager helped me: it let me “specialize” in systems and applications management… a field where you need to know a bit of everything anyway: infrastructure, security, logging, scripting, databases, and so on… πŸ™‚
This way, everyone wins.

Don’t misunderstand me, this does not mean I want to know everything. One cannot possibly know everything, and the more I learn the more I believe I know nothing at all, to be honest. I don’t know everything, so please don’t ask me everything – I work with mainframes πŸ™‚
While that can be a great excuse to avoid neighbours and relatives annoyances with their PCs though, on the serious side I still believe that any intelligent individual cannot be locked into doing a narrow thing and know only that one bit just because it is common thought that you have to act that way.

If I would stop where I have to stop I would be the standard “IT Pro”. I would be fine, sure, but I would get bored soon. I would not learn anything. But I don’t feel I am the standard “IT Pro”. In fact, funnily enough, on some other blogs out there I have been referenced as a “Dev” (find it on your own, look at their blogrolls :-)). But I am not a Dev either then… I don’t write code for work. I would love to, but I rarely actually do, other than some scripts. Anyway, I tend to escape the definition of the usual “expert” on something… mostly because I want to escape it. I don’t see myself represented by those generalization.

As Phil puts it, when asked “Are software developers – engineers or artists?”:

“[…] Don’t take this as a copout, but a little of both. I see it more as craftsmanship. Engineering relies on a lot of science. Much of it is demonstrably empirical and constrained by the laws of physics. Software is less constrained by physics as it is by the limits of the mind. […]”

Craftmanship. Not science.
And stop calling me an “engineer”. I am not an engineer. I was even crap in math, in school!

Anyway, what does this all mean? In practical terms, it means that in the end, wether I want it or not, I do get considered an “expert” on MOM and OpsMgr… and that I will mostly work on those products for the next year too. But that is not bad, because, as I said, working on that product means working on many more things too. Also, I can point to different audiences: those believing in “experts” and those going beyond schemes. It also means that I will have to continue teaching a couple of scripting classes (both VBScript and PowerShell) that nobody else seems to be willing to do (because they are all *expert* in something narrow), and that I will still be hacking together my other stuff (my facebook apps, my wordpress theme and plugins, my server, etc) and even continue to have strong opinions in those other fields that I find interesting and where I am not considered an *expert* πŸ˜‰

Well, I suppose I’ve been ranting enough for today…and for this year πŸ™‚
I really want to wish everybody again a great beginning of 2008!!! What are you going to be busy with, in 2008 ?

Facebook implemented a user.setStatus API!

Finally, you CAN change your Facebook status programmatically in a way that is supported!

Some months ago Christian discovered a hack to change your Facebook status. Some other people also used it and extended it. I also ported it to C# and made a winform using its unofficial method.
Suddenly after, Facebook asked us to take down the code, as it violated their terms of service.

It has taken a while, some struggles, but now they finally recognized the need for federated status, and implemented a user.setStatus API.

Twitter is the first to pick it up, so now you can update twitter and have your status propagate in Facebook!

Well done, guys!

When I’ll have some time I might think of rewriting my app using the SUPPORTED method, maybe finally writing that Live Messenger plugin… it would be nice πŸ™‚
When I’ll have time…

ITPro vs. Dev: there is no such a thing.

Dave Winer wisely writes:

[…] I’ve been pushing the idea that every app should be a platform for a long time, that in addition to a user interface, every app should have a programmatic interface. For me the idea came from growing up using Unix in the 70s, where every app is a toolkit and the operating system is a scripting language. Wiring things together is an integral part of being a Unix user. It’s why programmers like Unix so much […]

It is entirely true. The limits are blurry, IMHO. In the Unix world it is common to find full-fledged “applications” which have been written by the ground up by people that were doing SysAdmin tasks, and those “applications” are usually just… scripts. Simple shell scripts, or something more evolved (PERL, PHP, Python) it does not really matter.

I am so tired of the division traditionally made in the Microsoft world between “Developers” and “IT Professionals”. We evenΒ have separate sites for the two audiences: MSDN and Technet. There are separate “TechED” events: for”Devs” and for “IT Pros“. There are blogs that are divided among the two “audiences”…

There aren’t two different audiences, really. There are people, with various degrees of expertise. There is no such a thing as a “developer” if he doesn’t know a bit how the underlying system works. His code is gonna suck. And there is not such a thing such a “IT Pro” that builds and integrates and manages systems if he does not have the palest idea of how things work “behind the GUI”. He’s gonna screw things up regardless of how many step-by-step (click-by-click ?) procedures you spoon feed him.

That’s why automation and integration are best done by people who know how to write a bit code.

The PowerShell folk GET IT.

On this website we use first or third-party tools that store small files (cookie) on your device. Cookies are normally used to allow the site to run properly (technical cookies), to generate navigation usage reports (statistics cookies) and to suitable advertise our services/products (profiling cookies). We can directly use technical cookies, but you have the right to choose whether or not to enable statistical and profiling cookies. Enabling these cookies, you help us to offer you a better experience. Cookie and Privacy policy