Cisco UCS Just Made VMware VMmark Very Interesting

So, it's time for me to eat a little humble pie...  A little while back I posted how VMware's VMmark has become increasingly less valuable.  This wasn't a knock against the tool, it was a knock against the vendors all using the exact same configurations with some slight tweaks to stay on top.  Cisco UCS in particular took a middle of the road route by not using the things we really care about that make Cisco UCS unique, namely the Extended Memory Technology and the Virtual Network Adapter (Palo card).

Well, that time has come.  Cisco just published a VMware VMmark score based on Intel's new Westmere 6-core processors and.... wait for it...  the Extended Memory Technology and the Palo adapter!!

Here is the link, take a look.  None of the other major vendors have published Westmere processors (that I have seen) so I'm not sure how the scores will stack up.  But, Cisco has done something truly different, because of the nature of Cisco's disruptive technology; they have taken a new road based on their unique technology.  Kudos to them!

Some highlights of the report:

• Cisco is using the Extended Memory Technology, 192 GB in a 48x4GB configuration
• Cisco is using the Palo adapter and split one port as one vNIC to the all virtual machine traffic except the web servers
• The other port on the Palo adapter is split into 27 vNICs.  26 vNICs were presented to the 26 web server vm's.  I assume the 27th was used for the uplink out.
• I couldn't find anything in the report of where or how the vHBA's were used off the Palo adapter, I wish that would have been included.

Many will say, this is just a "Lab Queen"!! (A Lab Queen is an unrealistic configuration that is dressed up for a benchmark test and would never be used in the real world).  The answer to that question is of course it is!!  But, the fact that this is a DIFFERENT Lab Queen is very cool.  Using a configuration like this is something other vendors won't be able to accomplish and really highlights the ways in which UCS is different.

Cisco UCS QoS vs. HP Flex-10 vNICs in VMware

This post will be more conceptual than technical.  I recently was asked how Cisco's UCS &  HP's Flex-10 network design approaches affect vSphere designs.  Even though the industry is moving towards a unified 10GB fabric, there are different ways to move data through this big "pipe" and still ensure/prioritize delivery.  As you would guess, Cisco and HP approach this problem very differently.  Cisco takes a network centric approach to the problem and HP takes a server centric approach to the problem.

HP's Flex-10

HP Flex-10 takes a 10GB connection and carves it up into multiple virtual NICs.  The size of the "pipes" can be turned up and down to match the amount of bandwidth needed for the NIC.  Think of it as placing smaller pipes in the big 10GB pipe.  This approach is great for vSphere admins because the virtual switches in vSphere can be configured to look just like they did with a bunch of 1GB links into the server.  The transition to this technology is seamless for the vSphere administrator.  I'll borrow a diagram from Barry's awesome article on Flex-10.  If you haven't read it, please do!

What is the down side to this method?

The down side to this approach is by placing multiple pipes within the larger pipes, you have now placed a CEILING on how much data can pass through that particular pipe.  Let's say you present a 1GB vNIC to vMotion and during a vMotion it would be to your advantage to have access to more bandwidth.  Too bad, 1GB is all you will ever get.

 Cisco UCS's QoS

Cisco UCS uses a method known as Quality of Service (QoS).  Most of us "server guys (and gals)" have no idea what this is.  Here is how I have come to understand it.  If this is wrong, please correct me.  Network traffic is given a priority and this priority kicks in WHEN THERE IS CONTENTION on the network.  So, instead of smaller pipes inside a large pipe, you have more of a priority system in place to guarantee certain levels of service.  Think of this as a FLOOR model.  You can have as much as you want as long as everyone else gets their minimums (they get their quality/guarantee of service).  If something needs to spike and there is room, it can spike and then return to normal.  Here is a diagram of our Cisco UCS with traditional switches.  This isn't 1000v but you get the idea.  As you can see, two big 10GB pipes into the virtual switches instead of smaller pipes into multiple virtual switches.

As the vSphere administrator, this looks very different from my old multiple 1GB links into my multiple virtual switches!

What is the down side to this method?

At this time, QoS for Cisco UCS appears complex to configure and represents a shift in thinking for the vSphere administrator. 

How is the QoS implemented for Cisco UCS and VMware?

That is a very good question.  I can't seem to find any documentation on how to actually do this yet.  I'm sure there is a Cisco internal doc somewhere but I haven't found anything public that lays out the hardware that is needed (do I need 1000v or Palo for this, can I use a CNA and the standard switches?) nor have I found a "cook book" that documents how to properly make QoS happen in a vSphere environment.  I'm sure this will happen in time and if you have a link, please leave a comment!

Which is better?

It depends on your point of view and the comfort level of your team.  I can easily see advantages to both approaches.  One is easier to implement, the other appears to be a more elegant (but complex) solution.  Cisco has once again brought a disruptive technology to the table that can't be ignored.  What are your thoughts?

Virtualization Podcast Directory

I have been a Podcast junkie for years.  I've subscribed and unsubscribed to countless feeds over the years.  Up until recently, there has only a handful of podcasts with a virtualization focus.  I have noticed a nice uptick in this number recently.  So, in the interest of spreading the love to everyone out there, here is a list of the virtualization podcasts I am subscribed too.  As always, I'm behind on my episodes but I hope to catch up soon!  If you know of any others or if you have any additions or corrections, please let me know!

• My Podcast with Brian Gracely (sorry all, my site, I get to go first) - The Cloudcast
• VMware Communities Roundtable - Weekly VMware podcast hosted by the honorable John Troyer 
• APAC Virtualization Roundtable - For all those folks on the "other side of the world" that aren't able to make the VMware Communities Roundtable, hosted by Andre Leibovici
• Infosmack - Hosted by Greg Knierieman and Marc Farley
• Virtualization Security Roundtable - Hosted by Edward Haletky (Texiwill)
• Virtumaia Podcast - Hosted by Rich Brambley  and Marc Farley
• Mike Laverick's Chinwag - Hosted by Mike Laverick
• Gestalt IT Tech Field Day Podcast - Great Information from the Gestalt IT Field Day Sessions
• Official VMware View Podcasts
• Official VMware vSphere Podcasts
• HP Enterprise Storage TV - Calvin's (hpstorageguy on Twitter) podcast on HP solutions
• This Week in Virtualization - Tech Target's Weekly Podcast 
• DABCC Radio
• vChat Podcast
• Cloud Computing Podcast Directory
• vSoup

If anybody has any others, please send them my way and I'll try to keep this page up to date!

Looks like Marc is the Leo Laporte of the Virtualization Podcast world!

VMware Lab Manager Install Notes and LDAP Import

Setting up Lab Manager can be a little complex.  It isn't as straight forward as some of the other VMware products so I wanted to provide some tips and tricks to get it all up and running.

Things you will need prior to Lab Manager Installation

• In vCenter, create the datastores, virtual switches, and Resource Pools that you will need.  The Lab Manager (LM) install will detect them at install and this will make configuration MUCH easier
• Create all groups and users that you will need in either Active Directory or LDAP
• If you will be using ip pools, define a block of static ip's ahead of time!
• The Lab Manager server is currently a Windows 2003 based server. It can be virtual and on the same ESX hosts that it will be controlling.  If you do this, DON'T name it lab-manager.  If you do, you will get an error during installation because the install tries to create a folder in vCenter called lab-manger.  You will have to rename your virtual machine to proceed.  Also, you will need to change the speed of the vmxnet3 NIC per Jason's article here.
• Make sure both forward and reverse DNS lookup work between the vCenter server, LM server, and all vSphere servers
• The LM Server requires IIS 6.0 and .NET 2.0 to be installed.  IIS MUST be installed before .NET 2.0
• DON'T put the LM Server into the AD Domain.  VMware recommends against this even if you are importing users and groups from AD into the LM Server.  I asked why at Partner Exchange and I was told because it isn't needed and changes to AD could mess up the LM server.

LDAP/AD Integration

Integration with Active Directory or LDAP is the key to Lab Manager.  Lab Manager allows you to create single users on the box but NOT groups.  This makes security and configuration VERY difficult.  At the same time, the LDAP integration leaves a little to be desired in the implementation.  Here's how to do it:

• From the Lab Manager Interface, On the left hand side choose Settings and click the LDAP tab:

 

•  Once that is complete, you are ready to import groups.  Click the Import Groups Button.

 

•  Here's the magic.  Because the group and users have already been created in Active Directory, you can choose the group and assign it to the users role (the default role is read only so be sure to change it).  All users in this group are now Lab Manager Users

A few interesting notes about this import process.  If you look at the group once it is imported and no one in the group has logged in yet, the group appears blank!  This threw me for a little bit.  I expected it to populate with the users at creation time.  Instead the list populates at each USER's FIRST LOGIN!  My group has three users in it total.  As users log in, they will populate the group and also appear on the Lab Manager's Users list.  Here are a few screenshots as I logged in my test users.

Lab Manager Group with only first test user logged in:

Lab Manager Users Pane with two users created from login:

 Look for more articles as I get everything set up!

IBM's New eX5 Server Announcments

I wanted to tell everyone about the new server lines IBM announced today.  I attended IBM Business Partner training on this a few months ago and the products are impressive.  I was under NDA until today to speak about anything.  I can talk about the IBM products specifically but I'm still not able to talk about the Intel Nehalem EX chipset.  I will have in depth posts of the EX chipset when it is officially released.  Also, I am writing this from notes taken a few months ago so a few things might be slightly off.  If you see a mistake, please let me know and I will correct it!

As always, Kevin does an awesome job of laying out the products (and has some great pictures) so head over to his site for an introduction.  Hot off the press, Kevin has another article on just the X5 blade here.

Here's the basics:

• The servers contain Intel's yet to be announced Nehalem EX chipset.  I can't discuss the details on that since I'm still under NDA.  I will present what has been pre-announced by Intel.
• The Intel Nehalem EX (Intel 75XX) was designed by Intel to be the 4 socket follow up to the previous generation, the Intel 74XX.  This was the IBM 3850M2 and the HP DL 580 servers.
• (My opinion here, don't fuss at me Intel and IBM) Intel intended the Nehalem EX to be a 4 socket architecture.  IBM modified the architecture in cooperation with Intel for 2 socket servers.
• IBM has released the following servers based on Nehalem EX:
• 2 socket rack server called the x3690 X5.  It can hold two Intel 75XX processors and 32 memory slots
• 2 socket blade server called the X5 blade.  This was a pre-announce so I can't talk much about it yet.  One thing that will be cool about the blade is it will be "lego based".  By this I mean you can buy one and snap on another for a 4 socket blade
• 4 socket rack server called the x3850 X5 and the x3950 X5.  This will stack like the previous generation of 3850's and 3950's.  Each 3850/3950 will hold four Intel 75XX processors and 64 slots of memory
• Additional memory can be bolted on to any of the models above using an IBM exclusive attachment called the MAX5.  This will be a 1U (for the rack servers) with 32 memory slots or 1 blade width attachment that will give you an additional 24 memory slots.  It attaches directly into the Intel QPI (Quick Path Interconnect) bus for easy, low latency memory expansion of the models
• If I remember correctly both the 3690 and the 3850/3950 will have 1 GB on board network ports but an Emulex card can be added to the systems to replace the 1GB with 10GB on board

What do we know about the Intel EX chipset and why do we care?

I will point you to links here and here.  As I stated before, I'll have in depth analysis of the chipset when it is announced.  The why we care part is actually really cool.  There are some great advancements in the technology but there are also many things to make your life easier at time of purchase as well.

In conclusion, I'm very excited about the 2 socket offerings.  They appear to be very innovative and exciting.  I wasn't given access to any other vendor's early release information so I'm not even sure if anybody else will offering 2 socket servers based on Nehalem EX.  Interesting times indeed...