Cloud Provider

This is the third of a series of blog posts around Cloud Computing. It gives an overview about the current Cloud Computing trends and explains how to set up a private cloud environment with Microsoft's System Center products.

In this post the different cloud solutions currently available are compared. "Cloud" solutions which are not related to infrastructure are not considered in this overview. Meaning every Software-as-a-Service or central data storage solutions like Skydrive or iCloud are out of scope.
Following you find an overview about the main infrastructure and platform cloud solutions:

	VMWare Cloud Foundry	Microsoft Cloud	Amazon Elastic Compute Cloud (EC2) / Amazon Web Services (AWS) Elastic Beanstalk	IBM SmartCloud	Google AppScale
Description	Open-Source solution from VMWare written in Ruby	Public Cloud: Windows Azure, Private Cloud: System Center Suite	Amazon's cloud hosting solution	SaaS/PaaS/IaaS solution from IBM	Google's framework for developing highly scalable applications
Cloud Type	Private Cloud	Public and Private Cloud	Public Cloud	Public Cloud	Public Cloud
Service Model	PaaS	IaaS, PaaS	IaaS, PaaS	IaaS, PaaS	IaaS, PaaS
Virtualization	VMWare ESX	Windows Server Hyper-V, VMWare ESX, Citrix Xen	Citrix Xen	IBM	Xen, KVM, Eucalyptus
Supported Technologies	ASP .NET, Java, Spring, Ruby on Rails, Sinatra, Node.js, Grails, Groovy, Scala, PHP, Python/Django	ASP .NET, Java, PHP, Ruby, Node.js	Java, PHP, Ruby, Python, ASP .NET	Java, PHP	Java, Python, Ruby
Supported Application Servers	Java Application Server	IIS, Apache Tomcat	IIS, IBM WebSphere, Java Application Server, Oracle WebLogic Server, Apache	IBM WebSphere	(custom)
Supported Databases	Microsoft SQL Server, MySQL, Redis, MongoDB	Microsoft SQL Server	Microsoft SQL Server, MySQL, Oracle	Based on DB2 and compatible with Oracle	HBase, Hypertable, Apache Cassandra, MySQL Cluster, Redis
Operation Systems	Linux, Windows, custom	Windows, Linux, custom VHD	Windows, Linux	Linux	Linux
Price Model	Not published yet (Beta)	Pay-per-use	Pay-per-use	Price Table	Quota-based Pricing

Public Cloud Solutions
Currently, the main players in Public Clouds are Microsoft and Amazon. Amazon comes from an IaaS approach and Microsoft started with PaaS. These are also the areas in which each provider leads the market.
Amazon has in general more experience with Cloud Computing. Their public cloud offers great flexibility and full control over the infrastructure. On the other hand the patch management has to be done by the client. Amazon introduced AWS Elastic Beanstalk which provides a standardized automated deployment approach and the possibility to scale out applications and services easily. But the cloud consumer is still responsible for the underlying virtual machine and the update management.
Microsoft provides an extremely flexible PaaS solution which can be completely customized by invoking custom scripts. As long as installations and configurations are completely scripted and running on the windows environment, there are no limitations with this approach. Microsoft even takes over that patch management for the underlying operating system.

Private Cloud Solutions
In the Private Cloud sector there are at the moment just two alternatives: Microsoft's System Center solution and VMWare's Cloud Foundry. Microsoft is the only vendor which actually supports both, the private cloud and public cloud scenarios as well as a way to shift applications and services smoothly between private and public cloud data centers.
In the following posts I am going to explain in detail what Microsoft’s System Center Suite can offer and how to configure a private cloud environment.

Cloud Terminology

This is the second of a series of blog posts around Cloud Computing. It gives an overview about the current Cloud Computing trends and explains how to set up a private cloud environment with Microsoft's System Center products.

This post explains the key criteria, different approaches and service models of cloud computing.

Cloud Computing Key Criteria

• Elasticity and Flexibility
  The needed resources like CPU, Memory and Storage can be dynamically allocated. It is possible to quickly scale vertically and horizontally.
• Self-Service
  The cloud is exposed to the end-user in a way that they can easily control their needed systems and resources. There is no human-interaction needed for creating new virtual machines, deploying applications or scaling them.
  
• Completely Automated
  The most important criteria of cloud computing is the automation of the complete infrastructure. Systems and Applications are provided in an automated way where no human-interaction is needed.
  

Cloud Service Models

• Infrastructure as a Service (IaaS)
  This service model gives the user the possibility to provision virtual machines on operation system level. It is possible to deploy and run arbitrary software. The customer does not have any control of the underlying cloud infrastructure. But she can manage operating systems, storage and network components like firewalls and load balancers.
• Platform as a Service (PaaS)
  The customer can deploy applications on PaaS cloud solutions without having access to the underlying infrastructure like operating system, network or servers. The applications are restricted to the supported programming languages and technologies. PaaS solutions provide automated deployment processes, monitoring and easy horizontal scaling features (spreading multiple servers).
  
• Software as a Service (SaaS)
  This model allows the customer to use applications hosted externally, like Office and Web Storage solutions. The customer does not have any access to the cloud platform itself.
  

In my opinion the SaaS model is not directly connected to cloud computing because it could be mainly also provided without a cloud infrastructure. If you take a look at the key criteria above, none of them can be really applied to Software-as-a-Service solutions. That does not mean that some applications are running on a cloud platform to reduce operational overhead and have better flexibility and scalability. It is just not needed to provide this service.

Cloud Computing Types

• Private Cloud
  The infrastructure is running internally within a company. The applications itself can be exposed to the outside world but the self-service and administration is done within a company and the internal network.
• Public Cloud
  Public Clouds are available over the internet and can be used by other companies to host their applications and services. The main public cloud solutions are Microsoft's Windows Azure, Google's App Engine and Amazon's Elastic Compute Cloud. They provide usually a pay-per-use price model where the costs depend on how much traffic and resources are used.
  
• Hybrid Cloud
  Hybrid clouds are a mix of private cloud and public cloud infrastructures. They consist of company-internal and external infrastructure which builds a federation to establish a communication between each other.
  

In the following posts I am mainly talking about private Clouds and how they can be built within a company-internal data center.

How to create a Private Cloud?

In the next couple of posts, I would like to describe the current trends around Cloud Computing and how you can actually set up a Private Cloud environment.

First of all, it is important to clarify some terms which are used in the Cloud Computing context. Cloud Computing is used in many cases as a marketing buzzword and this causes a lot of confusion because everybody has another picture about cloud computing.

In this post (and in my opinion also in general) the term Cloud Computing represents an approach how a highly flexible, scalable and completely automated infrastructure can be build up in a data center. This is mainly achieved by virtualizing and automating the completely infrastructure. Additionally, cloud computing is about how this infrastructure can be made available as a self-service for end-users.

If you follow the advertisements around Cloud Computing you realize that it is more and more used for all web applications and web services which provide central storage for data and enable accessibility for a variety of devices, like laptops, tablets, mobile phones and TVs. The TelekomCloud or iCloud from Apple are examples for central data storages but they do not necessarily have a dynamic Cloud infrastructure to serve these services.

In the following posts I am going to explain how you can build up a private cloud based on Microsoft's System Center 2012 components.

- Post 1: How to create a Cloud?
- Post 2: Cloud Terminology
- Post 3: Cloud Provider
- ...

NDepend 4.0 available

The NDepend version 4.0 has been released. For all which do not know what NDepend is, take a look at my previous post.

NDepend is a great tool for static code analysis of .NET code. It is also available for Java "JArchitect" and C++ "CppDepend".

One of the main new features of the version 4.0 is the new query syntax which is based on Microsoft's LINQ. It provides a comprehensive way of analyzing your code.

The following example checks whether a base class uses one of its derivatives:

warnif count > 0 from baseClass in JustMyCode.Types
where baseClass.IsClass && baseClass.NbChildren > 0
let derivedClassesUsed = baseClass.DerivedTypes.UsedBy(baseClass)
    where derivedClassesUsed.Count() > 0
select new { baseClass, derivedClassesUsed }

This example is quite simple, but with these new query capabilities it is even possible to make more complex queries. One of the best examples you find here. These queries highlight namespace cycles and mutually dependent namespaces by using NDepend queries. These queries allow checking if the code follows a layered approach even on namespace level. Namespace cycles or mutually dependent namespaces cause usually a higher effort when changing certain code parts. The query allows to verify that and shows even suggestions which namespace should not use the other one based on the analysis which namespaces uses more types than the other one.

The image shows the mutually dependent namespaces. Furthermore it highlights that the namespace "TSTune.BL.DTO" should most probably not use the "TSTune.BL.Logic" because 11 types of DTO are used by Logic and just 1 type is used by DTO. There is a high chance that the use of this type is a mistake.

This example shows that the new query language of NDepend provides great possibilities to verify your code and identify bad parts. If these checks are performed as part of the continuous integration process, they can be easily avoided and result in the end to a much more maintainable solution.

JSAnalyse for VS 2012 has been released

I am happy to announce a new release of JSAnalyse. You can download the latest version on the codeplex project JSAnalyse.

The new release has a couple of improvements:
- Visual Studio 2012 support
- Multiple JavaScript dependency diagrams can be created
- Enhanced Caching mechanism to support bigger object graphs
- Detects even more static references between JavaScript files

For those who do not know what JSAnalyse is, read my previous post "JSAnalyse published on codeplex".

TFS 2012 Build Server Installation - Fails with error "System.FormatException: Index (zero based) must be greater than or equal to zero and less than the size of the argument list."

If you get during the TFS 2012 Build Server Configuration an error message with the following error message:

"System.FormatException: Index (zero based) must be greater than or equal to zero and less than the size of the argument list."

This is a bug in the Build Server configuration tool. You can fix it by turning on your Windows Firewall.

The configuration tool tries to check the firewall and add an exceptional rule for the build server port which causes an exception because the firewall is not running. Microsoft actually handles this exception, but within the catch block they try to write a warning message out which unfortunately causes another exception.

Here are the results by using reflector:

Assembly: Microsoft.TeamFoundation.Build.Config.dll
Name: Microsoft.TeamFoundation.Build.Config, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
Class: BuildServiceHostUtilities

private static void RemovePermissions(Uri baseUrl, bool deleteFirewallException)
{
    if (baseUrl != null)
    {
        string permissionedUrl = GetPermissionedUrl(baseUrl);
        try
        {
            ConfigurationHelper.FreeUrlPrefix(permissionedUrl);
        }
        catch (Exception exception)
        {
            LogWarning(Resources.Format("CannotFreeUrlPrefix", new object[] { permissionedUrl, exception.Message }));
        }
        if (deleteFirewallException)
        {
            try
            {
                ConfigurationHelper.DisableFirewallException(baseUrl.Port);
            }
            catch (COMException exception2)
            {
                if (exception2.ErrorCode != -2147023143)
                {
                    LogWarning(Resources.Format("FailedDeletingPortExceptionFor", new object[] { baseUrl.Port, ExceptionFormatter.FormatExceptionForDisplay(exception2) }));
                }
            }
        }
    }
}

The RemovePermissions method in the BuildServiceHostUtilities tries to add an exception for the Build Service port in the line ConfigurationHelper.DisableFirewallException(baseUrl.Port);

This causes an COMException which will be just logged as an warning. So far so good. But unfortunately the call for formatting the warning message gets just two parameters passed. If we take a log in the resources of the dll the "FailedDeletingPortExceptionFor" text has 3 parameters defined:

Resources.Format("FailedDeletingPortExceptionFor", new object[] { baseUrl.Port, ExceptionFormatter.FormatExceptionForDisplay(exception2) });

FailedDeletingPortExceptionFor=Failed to remove firewall exception {1} for port {0}. Details: {2}

This finally causes the "System.FormatException: Index (zero based) must be greater than or equal to zero and less than the size of the argument list.".

Feature Roll-Out

This is the fifteenth blog of a series of posts about the topics Continuous Integration and Continuous Delivery. It explains how important continuous integration is, to deliver software in short iterations to the customer with a high quality standard.

As funny as it sounds but one of the main problems about Continuous Delivery is the permanent delivery. It can be that one feature is not completely implemented yet and spans multiple releases to get ready. The main idea of Continuous Delivery is to split big requirements into smaller junks which still gives the user new possibilities. Unfortunately, this is not always possible and therefore the feature-flagging technique gets important.

Feature Flagging simply means that every big new functionality should be built in a way that it can be easily turned on and off like in the following listing shown:

public void PlaceOrder(Order order)
{
  var orderSystem = CreateNewInstance();
  orderSystem.Place(order);
}

public IOrderSystem CreateNewInstance()
{
  if (FeatureFlagManager.IsAvailable("NewOrderSystem"))
  {
    return new OrderSystem();
  }
  else
  {
    return new LegacyOrderSystem();
  }
}

Advantages of Feature Flagging
This approach gives a lot of advantages and great flexibility during the roll-out of a new version:

• Features can be switched on and off, even for a certain group of users if the feature flag component has been implemented to support it.
• A feature can be smoothly rolled out for a small group of users (like administrators, testers, people from a country, etc...) and does not affect the other users. Therefore some people can test the feature in the real world environment before it is available for public use.
• The roll-out can be done step-by-step. It can be coordinated and monitored what effects it has on the whole system regarding performance or usability. This approach is especially in web applications with many users extremely useful where the load cannot be simulated on a staging environment anymore.
• If any problem occurs the old variant is just one click away and there is no need for a big rollback with possible data inconsistency or loss.
• Furthermore the problems can be identified by a small group of users and do not affect all users at once which might cause an extreme increase of the support tickets.

Problems with Feature Flagging
Of course, the trade-off of this approach is that the design of new components has to be thought through.
The code for implementing feature flagging (e.g. if clauses, factories or attributes) should not spread around the whole code and make it much more difficult to maintain. If a feature has been completely rolled out, it should be even removed to simplify the code afterwards again.
The applied changes (e.g. database schema change) have to be compatible for both code parts. This has to be considered anyhow to support hot deployments where the application stays online during a deployment.
Additionally, the test effort is higher because both cases have to be tested as well as the possible dependencies between these cases.

But in the end I think that Feature Flagging and step-by-step roll-out is a really important concept which is worth to use in bigger web applications. It helps to reduce the risk of deployments dramatically.