Design Consideration and 12 Factor Apps

In the previous post we learnt about Platform-as-a-Service (PaaS) like what is Platform, need of using PaaS, Goals of PaaS and various service providers in the market. Now, we look at what are the design consideration and 12 factor apps for PaaS application. These information will be really helpful when we are designing a Cloud application.

  1. Codebase
  2. Dependencies
  3. Config
  4. Backing Service
  5. Build, Release, Run
  6. Processes
  7. Port Binding
  8. Concurrency
  9. Disposability
  10. Dev/Prod Parity
  11. Logs
  12. Admin Process
The above are the twelve-factors apps, which we should follow while designing the PaaS/Cloud application.

Codebase:

Code should be version controlled. We can track our code changes by using version control systems. Running code and code from version control has to have a one-to-one relationship. We can also deploy our Codebase to multiple deployment environments without any issues. So, we might have different environments like Development, Staging and Production.

Dependencies:

Dependencies are declared and isolated. This factor states that we never use System level packages with in our application. These dependencies like any kind of tools, libraries and even databases. Instead of assuming everything is there, we should define it in manifest.yml/POM.xml file that has the power to bring all need together.
For example, we want to use specific version of libraries for any of the technology instead of latest one. So, we can specify that libraries version. This allows to eliminate conflicts too.

Config:

Configuration is Stored in the Environment. Code should be separated from configuration as much as possible and we can provide the configurations inside the environment that we are running our application in.
For example, Connection string varies from environment to environment. So, that’s not good idea to declare in a static way in our code. The better way is to configure it in the environment and use.

Backing Service:

Backing services as attached resources. This factor is really an important to build an Cloud or any other application. Our application might have lot’s of services that uses like Databases, Queuing, Caching, Web services etc. These all we can access through network which will not differ from our local. Importantly we can attach and detach these services from our application at anytime we want. Only the difference between is, environment variables that we are running our application.

Build, Release and Run:

Build and Run stages are separate. Once we deploy our application in production, there is no come back. Only we can simply change our code in production. Some times it is not possible and even impractical in most cases. So we should have build and run stages separate by using unique identifier. So, if any problem occurs in production we can simply know the problematic version and easy to identify code.

Processes:

Application executed as Stateless processes. We should never assume that State or Processes are part of our application. They can go away at anytime and simply crash. Of course we can go and use state of the application sometimes, but these kind of information should be stored in a state-full backing store. With this way we could separate application logic from states and we can get back to our application where it has been crashed because we know where we left off.

Port Binding:

Services are exported via Port Binding. This is one of the most important factor of 12 factor app which simply states that our application must be self contained. It should not rely on a specific web server or a port. So application should be listening on a very specific port rather than 80, 100 and 44. Because these are the ports can be used for Caching mechanism from a Queuing mechanism.

Concurrency:

Application scaled out via Process Model. Process must be embrace as First class citizens in our application because they simply do everything. We have processes for listening Queue and making HTTP requests and listening coming request. These are all being handled by individual processes.

Disposability:

Processes are disposable. Minimal startup time and graceful shutdown is key here. Assume our application has been crashed. So, do you want it to recover back withing seconds or after few minutes? definitely want to recover Seconds of time because time equals to money.

Dev/Prod Parity:

Important point of Parity between application environments factor is to avoid time personal and through gaps between environment. We should be designing for a Continuous deployment pipeline. For example, developer want to run some sql scripts into production which is working in development. Suddenly it’s not working in production because Parity between the environments are missing. What about the time gap? should a developer go to the production on everyday or a month? if we shift to continuous, developer move code to production very often.

Logs:

Logs treated as Even Streams. Logs are very important aspect of running system because they can capture time order, chronological list of events happened through out the application life cycle. We treat them as Streams of events because they carry important information of application states. An application should never be concerned with storing it’s own log files. Logging framework must be taking care of where it has to collect them.

Admin Process:

Management task run as one-off processes. These task are important in any application to maintain and upgraded. So we should be using identical environments to running them. If our application is running in cloud, then we should run necessary scripts and tasks in the cloud.

For better understanding about the 12 factors of PaaS application use the below official website link.

The Twelve-Factor App

Anti-Patterns of PaaS:

These are few bad practices or Anti-patterns of the PaaS application.

  • We should not rely on local file systems
  • Scaling up services the old way
  • Trying to change the code on the server-side
  • Manually coordinating builds
  • Hard-coding configuration
  • Trying to be monolithic everywhere

What is platform and why we need PaaS?

What is Platform?

A platform is nothing but anything you can leverage to accomplish a task in a simpler/better way. As a programmer or developer we can take advantage of using existing code instead of writing from scratch. The most well known software platforms for Desktop applications are Windows, Mac OS. Even we can say some other examples to these explanation is Unix, Linux and Android, iOS etc.

A Computing platform or Digital platform is the environment in which a peace of software is executed.

Why we need PaaS?

In traditional IT decade, we have to manage everything that is like applications, OS, Servers, etc. So, we have to hire System Engineer to take care of our system and motor 24/7 of a week.

With the improved technology, we no need to worry about taking care of above all parts. There are some parts which are taken care by the service provider. Look at below image,

As per the above image, Virtualization, Servers, Storage and Networking are taken care by the Service providers. These kind of platforms are called as Infrastructure-as-a-Service (IaaS) and delivers to us by Amazon or Redshift.

With the Platform-as-a-Service (PaaS), we no need to worry about taking care of Software or Hardware. Those all taken care by service providers. All we need to do is develop our applications and services on top it. Lets look at below image,

Differences between Traditional IT, IaaS and PaaS:

Goals of Platform-as-a-Service (PaaS):

The ultimate goal of a PaaS is to make it easier for you to run your website or web application no matter how much traffic it get’s. It’s all about giving responsibility to run our applications and services properly with minimum downtime and issues.

We just deploy our application and service figures out what to do with it. Providers give us the guarantee that we can deploy our application without any problems. PaaS should handle scaling seamlessly for us so we can just focus on our application and code running it. We no need to worry about underlying details like, OS, CPU,etc which are taken care by platform.

PaaS Providers:

There are no of service providers available in now a day cloud market. Out of these, we can say the below few are biggest cloud service providers.

Internal process of deployment in Cloud Foundry

As of now we have learnt what is Pivotal Cloud Foundry and how to create application and push it into Pivotal Cloud Foundry. Now, we will look at what is the internal mechanism of an application deployment in Pivotal Cloud Foundry or we can also say that what is the cf push sequence mechanism.
As we already know, we need to use cf push command from CF CLI to push/deploy application into Pivotal Cloud Foundry. But most of them doesn’t know the internal process of pushing an application into Pivotal. Take a look at below sequence diagram to understand more about the deployment process of an application in Pivotal.

As shown in the image, there are few important stages which we will come as part of the application push is,

CLI

Cloud Controller

Blob Store

CC_DB

Cell (Staging/Running)

As shown in the above diagram, 12 steps are running internally to deploy an application into Pivotal Cloud Foundry. From step 1 to step 7 is part of Staging and 1 to 9 steps will part of Droplet of the application. Diego will be the responsible for running Droplet. Will look at each and individual steps which are show in above diagram,

  1. As shown in the image CF PUSH is the initial command which we need to issue from CLI to start to push application into Pivotal Cloud Foundry.
  2. Cloud Controller is the responsible for taking all the requests coming from CLI and similarly CF PUSH will come to Cloud Controller and create application which we are trying to push.
  3. Cloud Controller will send application METADATA to store into CC_DB (Internal Cloud Controller Database). METADATA is nothing but, application name, memory, buildpack and high level information.
  4. Application source files and other files related to pushing application jar/war will send to Cloud Controller.
  5. The RAW source code and application files are stores in the Cloud Controller database in the format of BLOB or Binary in Blob Store.
  6. Having all these application information in  place, Cloud Controller initiates the application startup process inside the container. So application start command is invoked and all the application startup events are triggered.
  7. With the help of Auctioner, try to identify which is the available CELL to stage the application. This CELL is also called as Staging Cell. As on when we issue the cf stage command, still this step will be executed internally.
  8. Streaming out put of the application staging will be send back to Command Line Interface (CLI) either it is success or any issues.
  9. After streaming out the application, Diego creates a TAR image of running application and stores in the format of BLOB. This is also called as droplet. Taking and running Droplet is the responsibility of Diego.
  10. After completed application staging, it report back to the Cloud Controller with the output information.
  11. Another Auctioner will happen to run the staged application. This auction will get identify the Running cell to run the application.
  12. Return the report of the running application back to the CLI (Command Line Interface)
What is the difference between Staging and Restart?
As shown in the image, staging will be in between 2 to 8 steps and 9 to 12 is the restart. Restart will work on top of already staged application