What is Cloud Storage?

Cloud storage is a model of networked online storage where data is stored in virtualized pools of storage which are generally hosted by third parties. Cloud storage is based on highly virtualized infrastructure and has same characteristic as Cloud computing in terms of agility, scalability, elasticity and multi-tenancy.

Cloud storage services may be accessed through a web service application programming interface (API), a cloud storage gateway or through a Web-based user interface.

• Cloud storage provides On-Demand storage space
• Cloud storage can be accessed through internet
• It’s pricing is based on Pay as you go and usage model
• Cloud storage platform exposes Rich application interfaces for communication purpose
• User need not manage the storage, everything is transparent to the user
• High degree of security and data redundancy and disaster recovery is built into Cloud storage
• Companies can focus on their core business and get rid of on-premise storage disks, maintenance tasks, backups, data replication, space and cooling requirements including human resource bandwidth.

How is it different from traditional storage?

• There is No difference with traditional storage techniques at functional interface level
• Traditionally there was need to pre-pay and buy storage device, Cloud Storage is delivered on demand
• Like traditional store there is no need of Capital Expenditures, Cloud storage is based on Pay as per used model
• Because Cloud storage is maintained by third party, it reduces CapEx and OpeEx significantly
• Traditional storage capacity use to be low because of upfront investment required, whereas in Cloud storage you get virtually unlimited storage capacity without upfront commitment/investment
• Traditional on-premise storage could not provide the level of high availability promised by Cloud Storage
• Unlike traditional storage, Cloud storage services are specialized for structured storage, unstructured storage, NOSQL data, large binary files, Media content streaming etc.
• Cloud storage services has data redundancy and disaster recovery built into it
• Cloud storage services are inherently build for high fault tolerance
• Since data will be stored in cloud storage which can be accessed from anywhere in the world with adequate security which gives kind of comfort for data users

Overall Cloud storage provides customers with required agility, scalability and cost effectiveness.

Economics of Cloud storage

While understanding the economics, it is necessary to try to compare apple with apple, generally organizations compare on-premise storage devices cost with cloud storage/GB, which will be wrong to do. For understanding actual cost required for X GB storage you need to consider below parameters.

• Upfront devices investment cost
• Operating Costs
• Redundancy cost
• Buy in advance to provide required capacity to the business
• Human resource involvement
• Disaster recovery
• Hardware support
• Technical trainings

Source: Forrester Research – based on sample data provided under specific conditions*

Challenges in using Cloud storage

Security

• Outsourcing data storage out of organization premises causes security risks increasing attack surface area.
• Security of data during transit

Accessibility

• Performance, reliability, available depends on WAN bandwidth between customer and Cloud Service Provider
• Although Cloud Service providers promises 99.9% availability, still it may prove problem for some organizations

Regulatory Compliances

• Data being stored in Cloud storage could be subject to government regulations and legal affairs e.g. Personal information or health care records
• Copyright and Piracy infringement for intellectual property

For e.g.

              1.The Patriot Act in the US allows the government to subpoena all data stored within the US. This might not be acceptable to many organizations.

              2. European Privacy Acts require that data be stored within the country of origin. Storing in the datacenter of out‐of‐country Service Provider cloud might not meet these requirements.

Cloud Service Provider stability

• Companies are not permanent and the services and products they provide can change. Outsourcing data storage to another company needs careful investigation and nothing is ever certain. Contracts set in stone can be worthless when a company ceases to exist or its circumstances change.

Total Cost

• Total recurring cost and bandwidth cost added up, may not prove feasible to some of the customers depending upon the size and data size and sensitivity to the business.

Many of the above concerns can be taken care by using Cloud storage Gateways.

What is Cloud Storage gateway?

For many organization’s may it be small-medium business or large businesses, majorly two obstacles stopping from utilizing cloud storage and those are really genuine but resolvable.

The first is the relatively slow performance as measured by response time obviously because of limitation of available internet bandwidth. This slow response time often makes cloud-based storage unacceptable for some users. The second is the requirement to write code for applications to the representational state transfer (REST) API. If the applications don’t have a native interface to cloud storage, many small to medium businesses lack the aptitude, desire, skills or time to develop it themselves.

So, what is the solution?

The potential solution could be to have a mechanism in place which will overcome above two obstacles and provide a way so that enterprises don’t need to worry about the storage, communication and performance complexities. A solution should provide wrapper to Cloud storage allowing users to use storage as normal as they had been doing on-premises.

‘Cloud storage gateways’ are the appropriate solution to this, since they are designed to overcome above two obstacles and they allow you to deal with cloud storage as if you are dealing with traditional SAN, NAS storage systems using NFS, iSCSI or FC methods. Additionally they can be used as a ‘primary storage’ unit providing features such as snapshots, thin provisioning, de-duplication and compression. This also eliminates any requirement of writing an application code for enterprises before using cloud storage.

There are many cloud storage gateways in the market, one has to understand how cloud storage gateway move your data cloud storage and how it brings back to you when needed. Various gateway companies use different approaches and algorithms some of them are patented technologies as well. The more efficiency achieved in data movement decides the quality and productivity of the cloud storage gateway. Additionally some cloud storage gateways allows you to use different cloud storage platforms such as Windows Azure, Amazon S3,  EMC Atmos, Nirvanix and others providing complete flexibility.

Basic working of Cloud Storage Gateways

Cloud storage gateways are nothing but a customized appliance (server) having various types and specialty disk storage in it like HDDs, SSDs (Solid state devices) and software control  on it. On-premise applications interact with these disks as normal, data is stored on these disks initially, it is them moved to the storage cloud based on policy/ algorithms such as age of the data, last access timings, or number of snapshots etc.

Though cloud storage gateways come with some costs but they relax you from several responsibilities providing low TCO solution. You get relax mind as cloud storage gateway takes care of data backups, snapshots, archival, de-duplication, compression, allowing you to use cost effective cloud storage, disaster recovery along with acting as a primary storage medium with classic storage technology combinations.

There are a number of cloud storage gateway vendors in the market today, with more emerging every quarter. They include Cirtas Systems, CTERA Networks, Nasuni Corp., StorSimple Inc., TwinStrata Inc. and others that are still emerging.

Usage scenarios for Storage as a Service

1.      Web Facing Applications

Opting out from owned on-premise infrastructure for a business application and moving it to the cloud could streamline the operations especially for data driven applications. Although, cloud storage can be used for any application dealing with data, it benefits specially when your data is increasing rapidly or existing data size is more than you want to control it on-premise. Cloud storage specially assists in web facing applications where upload and download of content is entirely up to end users and size of data can grow by any extent. Data could unstructured (simple files, documents, videos, audio, media content, database backups) or structured content (SQL databases) or NoSQL data, cloud storage is applicable for all kind.

Few examples of places where cloud storage can be used is-

• Media Streaming : streaming of audio & video
• Files/ document /photos/audio/ video storage e.g. drop box, You tube
• Store content for Social media sites like – Myspace, Facebook, Twitter, Blogs, etc
• Content Storage & Sharing : Pictures and content are stored in Cloud Storage e.g. Smugmug,
• NoSQL data storage

How about using cloud storage for on-premise web applications?

One can use cloud storage for on-premise application and it is perfect to use it. However, for better performance, Storage and application should be co-located in the cloud to avoid possible latency. REST API exposed by cloud storage services can be easily consumed by applications for leveraging services and reaping benefits like data redundancy, availability and cost effectiveness.

2.      Interfacing Smart Mobile devices

Mobiles are everywhere. Be it  be business phones like blackberry, Windows, iPhones or tablets/iPads with full blown applications generating and pushing data to central database for analysis, accounting and reporting purpose. Sales force or field agents and inspections generate lot of data with audio and video contents which need to be stored for longer period of process compliance purposes or reference purposes. Nowadays devices comes with inbuilt facilities to store content on local storage or on cloud, it has just become an option and cloud storage just a click away. Because of limited processing power, memory and bandwidth, data from mobile devices need to be pushed /pulled more frequently with availability requirement from anywhere in the world.

Cloud storage proves very efficient option in such cases providing complete data availability; data transfer rate doesn’t matter here since anyways it will happen via Internet with on-premise data center.

3.      Unstructured Data Storage

This is data is largest in size in any organization and uncontrollable also. People create copies of documents and version them as they want, it is difficult to track or control the pattern that users manage documents including emails, text documents, images/photos, manuals, training contents, proposals, marketing contents, accounting statements etc.

As per An IDC paper, “The Diverse and Exploding Digital Universe,” highlights how a single email with 1MB attachment when sent to four people consumes a total of 51MB of storage. (Source: “The Diverse and Exploding Digital Universe, An Updated Forecast of Worldwide Information Growth Through 2011,” March 2008, by International Data Corporation.) In other words, email suffers from attachment size limitations and is also an inefficient way of data sharing.

If constraints are applied on storage sizes, users tend to delete the content which may again create problems in accessing that in future. So to deal with such situations, strong storage policy is needed appreciating business need and impact of data availability for the organization. There should be a flexible way of data sharing which will increase collaboration in the organization along with an approach should prove cost effective and add value in terms of availability, disaster recovery, data redundancies, backups and versioning support.

Cloud storage helps you address all of above concerns fostering effective data storage, sharing, availability and pay as you storage option.

4.      Backup/ Retention/Preservation to Cloud

I have met with one customer who is into construction business from homes to business towers to ships and dams. Company operates in 7 countries following country specific policies and regulations for record keeping. Some of the countries like U.S and Canada need all the records for a construction project to be retained for 10 to 15 years. Data should be recoverable and available when it is required.

One may serve such request by regular maintenance of your storage policies and infrastructure considering backups, verifications, retention, duplication activities. Not only for the regulatory compliances but it is also important for every organization to retain, backup and preserve their data which is a real asset and outcome of thousands of hours of work.

Cloud storage takes all of your responsibilities of storing your data consistently for years you want to store. Outsourcing such as pain area will allow your IT to focus more on innovation and value added service for the businesses and not merely on record keeping purpose. Below are some of the instances where cloud storage has been used from some time –

• Backup for individual machine & laptops to cloud E.g Cirrustore
• Backup of cloud computing data like OS images Eg: vsphere, Openstack Swift storage
• Backup for one cloud provider to another – double sure it!
• Restore the data backed up in cloud- Retention period and Secure deletion purposes
• Data Preservation (is distinguished from Archive/Retention) in that the goal of preservation is to actively maintain the upkeep of information, most likely for long periods of time. E.g. Preserve Libraries & University Archives / repositories
• Archive data that is less sensitive to latency

5.      Databases in cloud

Databases, may those be SQL or NoSQL can happily reside in the cloud. Some of the Cloud service provides like Microsoft Windows Azure has SQL DB database provided in a ‘Database as a service’ mode. Microsoft has partnered with other database providers like MySQL to make it available on their cloud platform. There is a long list of databases supported on cloud platform like –

• Cassandra
• Bigtable
• Hbase
• Hypertable
• Neo4j
• mongoDB
• Azure tables

SQL Database can deployed in cloud could be your primary databases, backup copies or as a secondary data source purpose like reporting purpose. SQL database in the cloud are charged based on monthly database size.

NoSQL databases present very cost effective ways for managing data and there are very good example in the industry that how people are using it for storing large data sets in the NoSQL database like Azure tables with partitioning policies spanning data in multiple datacenters around the world.

6.      Cloud Storage for cloud computing

If you are using cloud computing for any reason, there is high possibility that you will fall in love with cloud storage. Most of the cloud providers have storage integrated with other offerings they have.

Whenever we use cloud platform like Microsoft Windows Azure either in IaaS, PaaS or SaaS mode, you are actually using Azure Cloud storage in some or other format.

One of the typical examples when people use Cloud storage is when they use IaaS offerings-

• VM Image Store :Image of the Guest OS which is made available to hypervisors for staring a VM
• Guest Auxiliary storage: Provision the storage space, at a given QoS, which the guest needs beyond the boot storage.
• People store entire deployment packages in cloud storage in PaaS mode for providing autoscaling feature to add more servers in response to traffic attracted by the application.

7.      Global Content Distribution in Cloud

Global content distribution is not a new mechanism to boost application performance when you have wide range of users around the globe and your application content is cached at many places nearer to the users for faster delivery to them. With the advent of Cloud computing, this mechanism has become more powerful with integrated support from the cloud providers like Microsoft has CDN(Content distribution network) feature as of Windows Azure platform, users has to perform few clicks to configure it for their application and your site will be transformed with tremendous boost in content delivery. With increased number of CDN nodes around the globe, latency and scalability issues are being addressed very proactively and easily.

8.      Scientific calculations and researchers to collaborate and discover

Scientific societies and researchers need large storage systems to store their simulations during their research. Not all scientists have liberty to buy storage systems and maintain them. Cloud storage provides efficient technique for them to use actually when required, pay for use and release the resources once results are drawn from the calculations. Cloud storage has been considered good candidate for storing content generated in digital movie production. Computer generated movie production generates huge data which need to be stored for short period of movie production may be for few months.

For e.g. movie like ‘avatar’ generated one petabyte or one million gigabytes of data which was stored using Microsoft digital asset management solution’ which could be today stored in Cloud storage with added benefits.

Hope this helps!  🙂

Laxmikant Patil