Virtualization has been around for YEARS, it isn’t new, it isn’t scary, it is just a matter of:
Running a base OS
Using said base OS as a platform to run other OSs on
Securing the base OS and the OSs running on the base.
Now – What is so impossible about that? In my opinion people (funny thing is, it is mostly IT people) are running around like chicken little.
Here is what you need to secure a cloud or virtualized environment…see if it sounds familiar.
If you want it to be publicly accessable, you need a firewall
You need a basic network infrastructure, including layer 2-3 switches, routers, VLANs, etc.
You need IPS/IDS, both at the firewall (I prefer to NOT put it on the outside of the firewall, but just inside of it)
File integrity monitoring
Backup or DR/BCP plans
Does this sound like something you have heard before yet? If not, and if you area an IT person or “C” level person, you should be shot with a BB gun a thousand times.
Now…I am not about to tell you HOW to implement the above controls, as I don’t know a single thing (unless I have done work for you) about your network. The good thing is, you probably have professional IT staff, information security staff, desktop staff and assorted other folks who CAN tell you how to do this. Granted there are some hurdles to overcome when you are working with in a virtual environment, but there is not rocket science or brain surgery involved. Though, if you have a good IT staff, they may be tempted to try their hand at some brain surgery, if they are worth their salt, they think about it often! :o)
Here is my opinion on the subject – Think about virtualization before you implement it. Have a group of people who know their sh*t when it comes to networking, systems administration, desktop support, software development, etc. Just like ANY project, the more thought you put into it and the more professional people you have to discuss things with, the better things will go.
A firewall is a firewall, it can block, filter, track and manipulate network traffic.
Encryption is the same in a virtual or physical environment, it obscures your data.
Networks work almost exactly the same in a virtual environment as a physical one, research it.
All the above being said, there are some things you will run into that require you to re-think some of the normal things you take for granted.
My point is this – If you currently have a good IT staff and you want to virtualize one facit or the entire infrastructure of your company’s IT, you can do it and you don’t have to go all “chicken little”.
So…I have been fairly absent from this place. I was working for a company who demanded quite a lot of attention. I have moved to a new position with another company and will be posting some interesting things about Web Application Firewalls and Cloud Security in the near future.
I have a good friend who will be guest blogging as well, prepare yourself for some good wise ass humor and poking fun at some people who really need it.
Thanks for reading!
Scott
So, the last couple of weeks I have been really busy testing out several “Cloud” or “Grid” operating systems. None of the were perfect, or for that matter, really ready for prime time for several reasons.
I use Amazon EC2 on occasion for development or to build out test machines with some super cool open source software that I just can’t wait to play with. It does well, but doesn’t really provide me with easy ways to keep the state of a machine, download an image of that machine while it is still running, or install software on it, shut it off and bring it back up with the same patch level, software installed, etc. as when I shut it down. I don’t want to have costs associated with it when I am not using it – and they don’t make that easy!
On to the show -
I fiddled, tested and cursed at three main distributions, they were/are:
1. Ubuntu Enterprise Cloud
2. XenCloud
3. Nimbula
Here is how I break it down.
1. Ubuntu Enterprise Cloud – Mainly marketed as a “Private Cloud” solution. I think Ubuntu has done an awesome job on their server and desktop OS distros, the Cloud however is the most convoluted, unorganized, undocumented thing I have messed with in several years! (they are not alone, as you will see below!)
Why you ask? Hell, I don’t know…Like I said, they have done an excellent job with the plain OS distros, but the Cloud is flipping crazy! I grabbed the .ISO of the 64 bit server, ran the install as it is documented here – - https://help.ubuntu.com/community/UEC/CDInstall
The network portion of the setup is extremely important – I can’t say this loud enough – DON’T THINK YOU CAN GET AWAY WITH ANY SHORT CUTS!!! YOU WILL REGRET ANY YOU ATTEMPT!!! Set it up with two network segments, the first can be a second LAN that you can access via your normal LAN, you will need it to be accessable in order to access the management interface. So, if you have a LAN segment that is already running a DHCP server and you can access all the systems on it, like a normal work place or home network…forget it! You will need to configure another interface on your switch or firewall (I ended up adding another interface to my firewall/router and creating a new LAN, then giving access to the existing LAN to it.) I suppose those who are more educated in VLANs could have setup a VLAN for the management side and given access to it via the regular LAN, but I didn’t have a good switch, so the VLANs were not clean and that caused problems.
I chose the CD install VS. the install from packages, because I am a simple person. I prefer to do it the easy way and get to work, if I had to go through what I did with this install on a normal Linux or Windows install, I would have given up computers way back in the early 90′s!
So – On to what was wrong/right. After I installed the cloud controller, walrus and storage controller on my first server (a Penguin Computing AMD based box with 8 cores and 8 gigs of RAM, and 2 TB of storage) I thought it was odd that the management network interface would use DHCP and not offer an option to do a static setup, but, hey, I am a n00b, so I figured it knew what it needed. I moved to the next box, the same hardware configuration – exactly the same – I installed the Node controller on it. The install started off bad. It couldn’t find the CC, so I put the IP address in, and it took off. Once it got to the “do you want to install updates” portion, I chose yes, and it tried…very hard…to grab updates. It couldn’t. So, a week of trouble shooting and I finally figured it out.
YOU CAN’T RUN A DHCP SERVER ON AN INTERFACE THAT GETS IT’S IP ADDRESS VIA DHCP! Man, do I feel stupid! Anyway, I got that figured out and he DHCP server still would not start. I whipped out some serious Google-Fu and found how to setup the dhcpd.conf file to serve up a range and set the gateway, DNS, etc. that were needed. The DHCP server started! WOOHOO…right? No…not sure why, but the bridge interface that they elude to in the documentation was not working properly. Got that figured out with the same Google-Fu, IPV4 Forwarding was the issue, and was off and running. The node controller got an IP address from the DHCP server running on the cloud controller, I could ping from the cloud controller to the node controller and vise versa. However…when I tired to do an apt-get update, it failed to get to the internet. I never did figure this one out, but managed to find a work around – apt-cacher NG! Install it on the cloud controller and it acts as a proxy for the node controller to apt repositories. It was amazing! I actually could update my system a week plus after I started the installation.
Once you have your system up and going and talking between themselves, you are going to want to login to the web interface and grab your keys. I used the firefox plugin for Amazon EC2, called HybridFox to manage the images and start and stop instances. It was quite anti-climactic. Same issues as Amazon EC2 where you can’t save a machine state without doing snapshots and converting those to AMIs. I did find a good site via Google that has a tutorial on how to resize the drives and save snapshots as AMIs, links will be at the bottom of this post. So…I decided to go grab XenCloud.
XenCloud -
I had high hopes for this one…it has Xen in the name, it has to be calming and go smoothly…right? Well, mostly.
I burned the .ISO to DVD and booted my afore mentioned Ubuntu cloud controller, with, I must admit, a great amount of angst! I had spent SOOOOOO much time on it! Anyway, it was off and running, then failed. Sad faces abounded! Come to find out, the Penguin Computing systems have NVidia RAID controllers in them and Xen doesn’t agree with them for some reason. So, I went and grabbed some Tums (for my heartburn!) and got back to it. I disabled the RAID controller in BIOS and removed the mirrors I had created and just had a JBOD setup. Xen happily took off, installed and I started on the second node with the same changes to the RAID as above. It went smoothly from there. Xen provides some very nice templates for several Linux distros as well as Windows desktop and server operating systems. I built a Windows 7 32 bit system, enabled RDP on it and was off to play. It was pretty cool, performance wasn’t stellar, but it was decent. I had 2 cores and 3 gigs of memory on the Windows machine. The display drivers are standard VGA, so the display wasn’t very pretty, but everything worked. I then built an Ubuntu system, installed NoMachine on it and was able to NX to it from pretty much anything in the house. They (NoMachine) provide a web interface that loads a java applet for the client that worked pretty good on my MacBook Pro. I didn’t look to see if any of the clients they give away for free would work on it. So, Xen was really the easiest one to setup and actually use.
Nimbula -
These guys are pretty new to the game as a company, but as a bunch of developers and smart guys, they have been in the game from the start. They hail from Amazon and helped to develop and push the Amazon EC2 cloud. Again…I had high hopes, I did try to remind myself that they are at version 1.1, but it didn’t keep me from dreaming of super fast installs and sleek interfaces!
The install – Went pretty well, I used the same two machines as above and threw in another SuperMicro box that has 8 cores (Intel ones!) and 8 gigs of RAM. It only had 250 gig of storage, but after reading for about 4 days, I decided it wouldn’t cause any issues with the installation.
Nimbula is a bit unique. When I first started paying attention to them, they offered the install as a Debian based .ISO that was a netboot server with the Nimbula software on it. By the time I got around to installing, it was now a CentOS based .ISO…hmmm…wtf? Anyway…I digress. I burned the .ISO, put it in a laptop ( as they recommend) and fired it up. It booted up and I edited the config file as required. The network setup for Nimbula is more simple. You just need a flat network segment (or two) to be running in no time. I setup a 172.16.0.x/16 on my firewall/router, gave it two IP ranges within that segment and VIOLA’! It worked! I ran into some more quirks with the NVidia RAID controllers, so I decided that it was probably CentOS that had the issue, as that is what Xen uses as well for the base OS.
The system took a bit of time to setup, as the laptop turned out to not be the optimum platform for running the PXEboot system from, I borrowed a smaller dual core system I had with gig interfaces in it and booted it up. Things went MUCH faster. There were a couple of bug/oddities, but overall, I give it a “B” for ease of setup…mostly due to the lack of a convoluted network being a requirement. I am currently building a few machine images to test on it, I tried to convert a few AMIs I had built for Amazon, but that failed like video games at a baby shower. I will post an update once I actually get a machine image booted up and loaded with more details on the requirements.
If anyone is interested in how I figured out/got around a lot of the issues with Ubuntu and Xen, post a comment and I will gladly either write another post that is entirely technical with all the bloody details, or provide a document of some kind that can walk you through the stuff that no one is willing to talk about.
So…my grades for ease of installation, ease of booting up my first machine image and managing the image afterwards are as follows:
1. Ubuntu – Low “D” for ease of installation, “Incomplete” for ease of network setup and Low “A” for ease of getting a machine image up and going.
2. XenCloud – High “B” for ease of installation, “A+” for ease of network setup and Low “A” for ease of getting a machine image up and going.
3. Nimbula – “C” for ease of installation, A+ for ease of network setup and “I haven’t finished yet” for getting a machine image up and going…I can already sense a “C” or lower for this, as I have had to spend more than 5 hours on it.
I was going to post links here at the bottom for those who wish to evade the Google advertising monster traps, but if you want them, I will simply export the bookmarks and send them to you…there are MANY!
So, unless someone has lots of time on their hands (no, I don’t have a life), then I don’t really think that Cloud is a worth while pursuit. Stick with VMware for now, they at least can be bent by your will and will provide free training if you buy, these guys, I wouldn’t trust any training they have for some time yet. It isn’t documented well enough and they don’t really have an interest in helping people who are small time or hobbyists.
Til next time – Scott
So…how would you accomplish this? I have been mulling this over for a few weeks now and have come up with a couple of options.
1 – Windows servers can be configured to talk to each other using IPSEC, so I have been considering this, however, not all of the systems are Windows, so it won’t really cover the issue.
2 – Setup a PKI using Windows 2008 R2, issue certs to all of the hosts that need to talk to the systems, import the certs on HP-UP, Solaris and RedHat systems and use IPSEC via the cert to communicate with everything.
Well…I have gone the route of building PKI and issuing certs to everything and it is a bit painful. Again, not bad in a strictly MS environment, but when you throw in UNIX, Linux and network gear, it gets difficult.
I would like to encrypt all communications between app servers and DB server, I suppose I could turn o Advanced Security in Oracle, do what ever with an ancient Sybase system and then do some more SSL with MSSQL. This would protect the communications and the data from sniffing, so perhaps this is the way to go…again, quite labor intensive and painful to implement. If anyone is reading this and has ideas, comments are open to signed up accounts.
Ubuntu has always found a way to get back on my computers! I really enjoyed playing around with it back when it was a fledgling OS that really was more of an idea than an actual operating system that did things like it does now. Desktop, Server, Laptop, Netbook and now C L O U D ! !
I know…you are probably thinking “where the hell has this guy been, Ubuntu has had Cloud for a while now”, right? Yeah, I heard about it at roughly the same time as the rest of the world, but I was in the middle of traveling about 75% of the time working as a Security Consultant and didn’t have time to play.
My first jump off the cliff into the world of “Cloud Computing” was when they were still calling it “Grid Computing”. I was an early user (it was still beta) of 3Tera. Those guys? Yeah, they rock like Slayer and the Pogues! I hooked up with Layered Technologies for a taste of their Virtual Private Data Center that ran on 3Tera and it was tasty! I built several load balancers, firewalls (very basic as they were at the time). I built a few instances and loaded up Zimbra and some other software to test it out. Geek Heaven!
Now I am working in a “Normal” 9-5 job, so I have more time to drink from the fire hose. I installed Ubuntu Cloud on three Penguin Computing servers, dual quad core AMD Opterons with 16 gigs of RAM and 2 terabytes of disk. SCHWING!!!!!!!!!
Next I am going to work on building some templates for Ubuntu desktop systems and use No Machine to reach out and grab a desktop on them to see how they perform.
I am sure some of you noticed I didn’t elaborate on the build process of Ubuntu Cloud…well, there are a million places you can read about how to do that, I will document some of the more archaic stuff that takes some actual work.
More to come.
In the next few weeks, I will be writing about my adventures in implementing several Security Tools. I am a Network Security Engineer/Architect and enjoy a good challenge, so I pick some obscure stuff to play with.
The company I work for does not endorse my activities or comments. Some of the things I talk about will be setup for my employer, some will be side projects and yet others will be me geeking out at home in the lab.
I am a very opinionated person, so I will be hard on some things, and really love others…very seldom will I be in the middle on something.
My first two articles will be on the following:
AlienVault – OSSIM: AlienVault is an extremely good product, I managed to pull an outsourced solution based on Snort out and replace it in a matter of 3 months with OSSIM, or AlienVault Pro SIEM. I know a lot about the underlying technologies, but they REALLY know a lot and are an excellent company now that they are selling a commercial product.
Imperva Secure Sphere – Both the Database protection and the newer File Share Monitoring.
Suricata – Supposed to replace Snort (or compete with it anyway, I have my doubts if Snort can ever be replaced!).
More to come…
