Business Agility and Windows Azure Cloud Platform


IT contribution to Business agility using Cloud Technology

This white paper describes what is business agility, how it can be improved using IT Agility and role of Cloud computing on IT and business agility. “Agility typically refers to the ability of an organization to rapidly adapt to market and environmental changes in productive and cost-effective ways and meet customer demands by adjusting to the changes in a business environment by taking advantage of available resources”. Here, the word “resources” carries significant importance as it indicates what can change the entire game in organizations operations.  Agility and competitiveness goes hand in hand and is key factor in success of any business. As an Organization, who does not believe in agility is supposed to be leaping towards the obsoleteness and will be out of business in no time. Consider below 6 attributes which majorly influence the business agility in typical case.agility

An Organization can acquire agility by institutionalizing above 6 pillars in every possible business function of it. And IT is considered as one of the most important strategic business function of modern industry. IT becoming agile adds higher percentage of agility to overall organization. In this paper we will try to understand how IT agility impacts business agility. We will discuss this in the context of Microsoft Windows Azure Cloud computing as a platform to make your IT more agile, cost effective, efficient and flexible.

IT agility and business agility

Agility attributes

Focus on Time to value:

There is an almost obsessive focus on getting new products/services/features to market as soon as possible.  This enables a shorter feedback cycle and, equally importantly, a quicker timeframe to earn a return on the investment made. One can influence the market and competitor with this attribute.

cloudIT Response using Windows Azure Cloud Platform

  • Provision IT resources (Processing power, storage, bandwidth, databases, O.S etc.)  in minutes. No procurement review/approval cycles, negotiations, shipping delays, escalations. Windows Azure provides resources as quickly as you can imagine. Hit the market with your innovative ideas/ products in least possible time.
  • Tools and frameworks support to rapid application development shortening time to market. Windows Azure platform provides all the tooling and frameworks for your team to help build applications quickly.
  • Respond to your customers/needs immediately with added capacity, Windows Azure powers you with scalability and elasticity to allocate resources on the fly automatically as much you need any time. You can double or multiply the data/application load capacity in automatically.
  • Achieve quicker, faster and secure integrations with customers, partners, and vendors. Windows azure platform provides secure way to connect on-premise applications for data exchange with best in class security standards.
  • Expand your business around the globe. IT will follow you everywhere in no time. Windows Azure has data centers around the world providing lowest latency by deploying applications near to customer locations.
  • Gather data intelligence and analytics for taking brave decisions quickly, enable your application with cloud based reporting, Big data options to provide business predictions
  • Collaborate with stakeholders, provide right information at right place for right people using SharePoint Online or with any Content management and collaboration software

Innovation:

Always look for better way of doing things using latest and better technologies. This is considered as the most significant attribute and can impact to larger extent on enterprise’s success.

cloudIT Response using Windows Azure Cloud Platform

  • Cloud technology is the future, early adopters will benefit from it, all the innovations impacting business and IT will happen on this platform, future proof your business with this technology before your competitor does.

Flexibility: Flexible processes and options

This is where a lot of businesses (particularly larger enterprises) find it very difficult to move quickly to exploit new opportunities.  The more each business process is intertwined (coupled) with others, the harder it is to implement change quickly.

cloudIT Response using Windows Azure Cloud Platform

  • One of the reasons Cloud technology exists today is to provide required flexibility to business which was difficult to achieve earlier. Windows Azure platform provides numerous features and resources and each resource is available for use on Pay per use model. No capital expenditure is needed. No commitment to use resources is needed. Businesses can leverage Cloud resources on clock hourly basis and release them any time when no required. This flexibility will bridge the gap in large and small businesses where large business use to rely on their IT capacity to beat the small business competitors.

Low Latency: Faster decision making

The quicker a decision can be acted upon, the sooner the business knows whether the decision was right and what else needs to be done to achieve the underlying goal.

cloudIT Response using Windows Azure Cloud Platform

  • Once you sign up for Cloud, you will not face any challenges in procurement of resources like we do in on-premise case. No need to worry about Operating system software’s, Database software licenses, Server hardware, Network hardware, storage devices, load balancing mechanisms, back up mechanisms, disaster recovery mechanisms, failover sites, antivirus software’s, hardware support contracts, trainings, SLAs, WAN optimizations, routers, switches, power UPS, cooling, storage space, Blade/Rack Servers and what not. Cloud eliminates the decision making and waiting time delays.

Economic efficiency: Cost effective processes and resource leverage

A lazy, bureaucratic organization will protect its turf at the cost of progress and innovation.  Lean operations that minimize waste have the added incentive of promoting new and innovative ways of doing things, and being able to implement them quickly.

cloudIT Response using Windows Azure Cloud Platform

  • No upfront capital investment in any hardware, software’s, licenses allows you to try more business ideas with the market and fine tune your idea in response. This not only saves capital expenditure but also provides economic value in terms of Pay as you use model.
  • Reduce overall Total cost of ownership by eliminating CapEx and controlling OpeEx smartly.
  • If you don’t want 5 servers out of 7 in the night time, just remove them instantly and save money
  • With the help of Windows Azure platform support, Man to server maintenance ratio will increase allowing IT admin to manage, monitor more Cloud deployed Servers saving time which otherwise can be spend on innovative ideas which can benefit customers and prove to be differentiator in the market
  • If you are sure that you need certain resources for at least 6 months, you can opt commitment based plan and gain discounts up to 20%

Rapid Adaptation

This component really refers to the mindset of continual monitoring of, and adaptation to, changing market conditions.  It is the ultimate feedback mechanism that ensures the product/service offering is continually refined to best meet the needs/constraints of customers, suppliers and partners.

cloudIT Response using Windows Azure Cloud Platform

  • With an advent of cloud computing technology, you save lot of time in procurement, IT resource management, training, managing peak loads, licensing policies, support contracts etc. This allows you to focus more on your customer needs and studying market trends. This study will empower you to add more value in your business and correcting, fine tuning ideas, processes and methodologies regularly.
  • Once you are sign up for Windows Azure platform, all the inventions happening in the platform will follow your way which will help you further optimizing resources, adding more features to your applications and get feedback on it from customers to fine tune your offerings.

what can cloud do

Business Requirements and solutions

example

Company Profile for the Case study:

Dream medico equipment’s is public limited company having 2000 employees with annual revenue of $600 million. Company primarily manufactures innovative medical devices for customers around the globe which are majorly hospitals, pathologies spanning public, private and military sectors. Company has captured good market in America, Europe and Asia and is also planning to develop low cost medical devices/Products for personal use at home. Company has got many market competitors selling products at aggressive rates and providing good quality support services. As of now, IT department which manages all the application ecosystem on-premises including core JD Edwards ERP product. Through its lifespan company tries to solve above mentioned problems to be more agile and competitive. Let us see how IT agility will help company in achieving certain goals.

Revenue Growth

Problem statement: Dream medico equipment’s has been reviewing its revenue growth since last two quarters and it was found that it is satisfactory. However, company management thinks that if they focus more on possible growth areas they can increase revenues further.

Company higher management decides to be little aggressive in introducing new products in the market which was supposed to be launched one quarter later under personal use product category.

Company wants to prelaunch few innovative products with attractive price portfolio and sees how market responds to it. Company wants to book good amount of business during pre-launch offer and collect feedback about the sample products and Product specifications in accordance with prices quoted. Also customer, partners and vendors may provide feedback in terms of few new features they want to see in the products or entirely few new product ideas may take birth.

Dream medico equipment’s CEO calls for a meeting with other executives to decide on feasibility to conduct prelaunch event in months’ time and seek if one month is sufficient for the preparation. CEO explains significance of this prelaunch offer for forthcoming projects in terms of below benefits -

Business Benefits

  • Booking a business in advance
  • Market capture ahead of competitors
  • Brand establishment
  • Seek customer feedback on products and go back to Product design team and incorporate wherever applicable
  • Reduce risk of mass producing products if certain products are not well received
  • Look for big orders by health care institutions and care homes

Sales Head explain about the possible sales channels for Product promotion and accepting prelaunch orders. He wants to open up multiple sales channels to promote the product.

Business head explains which products to be launched with which features. Since this event will be driven and owned by business. Business head has determined some budget for it, when he discloses the budget and the date for the event which is just one month away. Other executives are not happy with available time, required preparations and budget which is they feel is very constrained.

Business head reiterates the significance of this event and also explains why there is definite budget available, he explains – “Since this event can empower our revenue growth, event has significance in time which cannot be postponed. Since this is just a onetime event and not an ongoing activity there cannot be more budget available for this. Also if products are not received well by customer, all these expenses will become waste and also it may impact on company brand in the market, so management doesn’t want to spend very big numbers for this event.”

Business Risks

  • Available time – The entire event may fail if preparations are not completed well in time. This included efforts by teams like marketing, sales, IT and Products team.
  • User experience – New product site may have performance issues because of unpredictable number of customers, partners, and analysts can visit from around the world. As site will be visited, videos will be downloaded, product documents will be downloaded, adverts will be played, orders will be accepted, and payment transactions will be done.

As usual, IT head was under pressure to provide the required support for this business initiative. After further discussions with business, marketing and sales team, it was decided to build below systems for specific purposes.

  • Web Sales Channel – Major sales channel will be a new web site where all pre-launch products will be listed along with prices, estimated arrival time, features and specifications, videos, adverts. Customers can visit the site, review products and put comments /feedback against it. If they like a product they can place an order. This site should be separate from existing company product web site. However, there should be feature in existing and new site to allow users move back and forth.
  • Second sales channel will be a tablet application for sales team who will be in the field in various hospitals and health care institutes. Sales team will be demoing products and showing videos, adverts on the tablet along with accepting orders through tablet application.
  • Business team needs a light weight reporting tool for understanding business realization of this event. This reporting application will not be needed after this event unless another product launch in future.
  • Company wants to add new products into their existing distributor’s portal so that they can place orders for new products. Business estimates that there will be large traffic attracted by this portal because of distributors accessing it concurrently. IT needs to take care of appropriate performance needs during peak time.

Major challenges in front of IT manager are –

1. Development of –

  • New web site for web sales channel in short period of time
  • Tablet application for sales team
  • Reporting website

2. Manage high traffic demands on all of above sites

3. Provision hardware, network and software resources in available time and available budget

4. Delays in procurement of resources(hardware, software)

5. Keeping total cost of ownership low, avoid upfront investment as resources are not needed for ever

6. Look for some innovative way of achieving all these challenges to serve such requests in future

During this analysis, IT head and Manager decides to take a next step and concludes to select Microsoft Windows Azure Cloud Platform as a future platform not investing any more in on-premise data center.

Cost Reduction

The first challenge of budget limitation is addressed by choosing Windows Azure platform.

  • No upfront cost investment in software’s, hardware, licenses, failover sites, storage disks, bandwidth, Racks/blades, load balancers etc.
  • Company need to pay as per your usage only
  • Cost can be controlled at hourly unit basis
  • Stop paying for resources released back to platform
  • Company can also save from other costs involved in on-premise deployments like electricity power, cooling, Space
  • Human resource time saving which is usually required to monitor, manage and maintain the servers
  • Better human-to-server maintenance-ratio for managing cloud deployed servers saving administrators savings
  • Time saving in Procurement, decision making, reviews and approvals
  • Time saving in maintaining Server O.S and database licenses, service packs/patches, license expiries, hardware compatibility, CALS etc.

Manage Business in Peak Period

Major challenge in managing business in this period was lack of information on potential load or unpredictable nature of traffic spike. Team needed really dynamic scalable solution to auto adjust the server resources per actual traffic need.

Azure Platform inherently supports scalability and elasticity in cost effective manner. Team implemented auto scaling solution to increase number of servers as load increased and decrease number of servers during low traffic time. All this process can be done automatically without manual intervention.

Company could reduce lead time completely by providing adequate number of resources required for the event resulting in best user experience and successful sales trial.

Response to Risk

With the help of Windows Azure technology, IT department could deal with Business and IT risks in below way-

  • Using Windows Azure platform support, tools and frameworks, websites were developed in rapid way. Company opted for cloud enabled ‘NopCommerce’ open source framework to quickly develop the new web site. This framework has features already available required for this website.
  • Since ‘NopCommerce’ framework like other numerous products is pre-integrated into the Azure platform. This allowed development team to quickly develop the websites and test it by deploying it in Azure staging environment.
  • For storing video, audio, images, product specifications document, team used Windows Azure Blob storage services. This is very cost effective and efficient option.
  • For managing another risk of user experience under such a high load condition was managed by -
    • Better video and audio streaming Video/audio content is stored in ‘Block blob’
    • Since Site was supposed to be used across the globe, team used Azure Content distribution Network (CDN) for distributing and caching content among various Azure nodes for best performance.
    • Using Azure ‘Auto scaling’ feature, sites could be dynamically scaled out to required number of servers to server the traffic

Thus, IT agility could help manage business risk effectively.

Low Cost Reporting Solution

Development of reporting site was done with least amount time by developing few reports using SQL Server Reporting services feature. Team needed web server to deploy the reports on-premises, which was time consuming process to get. Team used SQL Azure Reporting services for report deployment which is charged per number of reports, which also eliminated the need of buying SQL Server license. After the event, reports will be removed from the SQL Azure Reporting service server stopping getting charged further.

Sales team Mobility

For Tablet application development, Windows Azure Mobile Services proved to be accelerator where team just designed a database and platform automatically creates required template web services and sample client mobile application specific to platform such as windows Phone, IPad/IPhone etc. Team could save considerable time in development and testing phase, Team did not experience any environment differences (Production/Staging) and effort wastage cause of it. Team could gain required scalability in no time when needed.

Sales - Distributor Connect

Anticipating high load on Sales-Distributor portal, entire portal was moved to Azure platform. Team used IaaS feature of platform by provisioning Windows Server 2008 R2 Virtual machine and another virtual machine with SQL Server 2008 R2. Since this portal was built for scalability, team could scale this portal on multiple web servers in load balanced manner which is implicitly available in Azure platform. Team need not bought O.S or SQL Server licenses for this migration which is already considered by the platform in per hour pricing.

last image

Friends ! I hope this helps :)
Laxmikant Patil

 

 

Windows Azure Technology Partner Selection Guide – white paper


Dear All,

My white paper on “Windows Azure Technology Partner Selection Guide” has been published on company web site at http://www.kpitcummins.com/application-and-consulting-services/application-services/microsoft/windows-azure

Please visit this page and click on “Download White Paper” link.

Or

Paper can be downloaded directly at http://www.kpitcummins.com/downloads/whitepapers/windows-azure-partner-selection-guide.pdf

Enjoy reading it !! :)

 

Thanks

Cloud Storage Use Case Scenarios – Considering Windows Azure Storage Platform


What is Cloud Storage?

Cloud storage is a model of networked online storage where data is stored in virtualized pools of storage which are generally hosted by third parties. Cloud storage is based on highly virtualized infrastructure and has same characteristic as Cloud computing in terms of agility, scalability, elasticity and multi-tenancy.

Cloud storage services may be accessed through a web service application programming interface (API), a cloud storage gateway or through a Web-based user interface.

  • Cloud storage provides On-Demand storage space
  • Cloud storage can be accessed through internet
  • It’s pricing is based on Pay as you go and usage model
  • Cloud storage platform exposes Rich application interfaces for communication purpose
  • User need not manage the storage, everything is transparent to the user
  • High degree of security and data redundancy and disaster recovery is built into Cloud storage
  • Companies can focus on their core business and get rid of on-premise storage disks, maintenance tasks, backups, data replication, space and cooling requirements including human resource bandwidth.

How is it different from traditional storage?

  • There is No difference with traditional storage techniques at functional interface level
  • Traditionally there was need to pre-pay and buy storage device, Cloud Storage is delivered on demand
  • Like traditional store there is no need of Capital Expenditures, Cloud storage is based on Pay as per used model
  • Because Cloud storage is maintained by third party, it reduces CapEx and OpeEx significantly
  • Traditional storage capacity use to be low because of upfront investment required, whereas in Cloud storage you get virtually unlimited storage capacity without upfront commitment/investment
  • Traditional on-premise storage could not provide the level of high availability promised by Cloud Storage
  • Unlike traditional storage, Cloud storage services are specialized for structured storage, unstructured storage, NOSQL data, large binary files, Media content streaming etc.
  • Cloud storage services has data redundancy and disaster recovery built into it
  • Cloud storage services are inherently build for high fault tolerance
  • Since data will be stored in cloud storage which can be accessed from anywhere in the world with adequate security which gives kind of comfort for data users

Overall Cloud storage provides customers with required agility, scalability and cost effectiveness.

Economics of Cloud storage

While understanding the economics, it is necessary to try to compare apple with apple, generally organizations compare on-premise storage devices cost with cloud storage/GB, which will be wrong to do. For understanding actual cost required for X GB storage you need to consider below parameters.

  • Upfront devices investment cost
  • Operating Costs
  • Redundancy cost
  • Buy in advance to provide required capacity to the business
  • Human resource involvement
  • Disaster recovery
  • Hardware support
  • Technical trainings

Source: Forrester Research – based on sample data provided under specific conditions*

Forrester Internal Storage

Forrester Cloud Storage

Forrester Cloud Gateway Storage

Challenges in using Cloud storage

Security

  • Outsourcing data storage out of organization premises causes security risks increasing attack surface area.
  • Security of data during transit

Accessibility

  • Performance, reliability, available depends on WAN bandwidth between customer and Cloud Service Provider
  • Although Cloud Service providers promises 99.9% availability, still it may prove problem for some organizations

Regulatory Compliances

  • Data being stored in Cloud storage could be subject to government regulations and legal affairs e.g. Personal information or health care records
  • Copyright and Piracy infringement for intellectual property

For e.g.

              1.The Patriot Act in the US allows the government to subpoena all data stored within the US. This might not be acceptable to many organizations.

              2. European Privacy Acts require that data be stored within the country of origin. Storing in the datacenter of out‐of‐country Service Provider cloud might not meet these requirements.

Cloud Service Provider stability

  • Companies are not permanent and the services and products they provide can change. Outsourcing data storage to another company needs careful investigation and nothing is ever certain. Contracts set in stone can be worthless when a company ceases to exist or its circumstances change.

Total Cost

  • Total recurring cost and bandwidth cost added up, may not prove feasible to some of the customers depending upon the size and data size and sensitivity to the business.

topic

Many of the above concerns can be taken care by using Cloud storage Gateways.

What is Cloud Storage gateway?

For many organization’s may it be small-medium business or large businesses, majorly two obstacles stopping from utilizing cloud storage and those are really genuine but resolvable.

The first is the relatively slow performance as measured by response time obviously because of limitation of available internet bandwidth. This slow response time often makes cloud-based storage unacceptable for some users. The second is the requirement to write code for applications to the representational state transfer (REST) API. If the applications don’t have a native interface to cloud storage, many small to medium businesses lack the aptitude, desire, skills or time to develop it themselves.

So, what is the solution?

The potential solution could be to have a mechanism in place which will overcome above two obstacles and provide a way so that enterprises don’t need to worry about the storage, communication and performance complexities. A solution should provide wrapper to Cloud storage allowing users to use storage as normal as they had been doing on-premises.

‘Cloud storage gateways’ are the appropriate solution to this, since they are designed to overcome above two obstacles and they allow you to deal with cloud storage as if you are dealing with traditional SAN, NAS storage systems using NFS, iSCSI or FC methods. Additionally they can be used as a ‘primary storage’ unit providing features such as snapshots, thin provisioning, de-duplication and compression. This also eliminates any requirement of writing an application code for enterprises before using cloud storage.

There are many cloud storage gateways in the market, one has to understand how cloud storage gateway move your data cloud storage and how it brings back to you when needed. Various gateway companies use different approaches and algorithms some of them are patented technologies as well. The more efficiency achieved in data movement decides the quality and productivity of the cloud storage gateway. Additionally some cloud storage gateways allows you to use different cloud storage platforms such as Windows Azure, Amazon S3,  EMC Atmos, Nirvanix and others providing complete flexibility.

Basic working of Cloud Storage Gateways

Cloud storage gateways are nothing but a customized appliance (server) having various types and specialty disk storage in it like HDDs, SSDs (Solid state devices) and software control  on it. On-premise applications interact with these disks as normal, data is stored on these disks initially, it is them moved to the storage cloud based on policy/ algorithms such as age of the data, last access timings, or number of snapshots etc.

Though cloud storage gateways come with some costs but they relax you from several responsibilities providing low TCO solution. You get relax mind as cloud storage gateway takes care of data backups, snapshots, archival, de-duplication, compression, allowing you to use cost effective cloud storage, disaster recovery along with acting as a primary storage medium with classic storage technology combinations.

There are a number of cloud storage gateway vendors in the market today, with more emerging every quarter. They include Cirtas Systems, CTERA Networks, Nasuni Corp., StorSimple Inc., TwinStrata Inc. and others that are still emerging.

Usage scenarios for Storage as a Service

1.      Web Facing Applications

Opting out from owned on-premise infrastructure for a business application and moving it to the cloud could streamline the operations especially for data driven applications. Although, cloud storage can be used for any application dealing with data, it benefits specially when your data is increasing rapidly or existing data size is more than you want to control it on-premise. Cloud storage specially assists in web facing applications where upload and download of content is entirely up to end users and size of data can grow by any extent. Data could unstructured (simple files, documents, videos, audio, media content, database backups) or structured content (SQL databases) or NoSQL data, cloud storage is applicable for all kind.

Few examples of places where cloud storage can be used is-

  • Media Streaming : streaming of audio & video
  • Files/ document /photos/audio/ video storage e.g. drop box, You tube
  • Store content for Social media sites like – Myspace, Facebook, Twitter, Blogs, etc
  • Content Storage & Sharing : Pictures and content are stored in Cloud Storage e.g. Smugmug,
  • NoSQL data storage

How about using cloud storage for on-premise web applications?

One can use cloud storage for on-premise application and it is perfect to use it. However, for better performance, Storage and application should be co-located in the cloud to avoid possible latency. REST API exposed by cloud storage services can be easily consumed by applications for leveraging services and reaping benefits like data redundancy, availability and cost effectiveness.

2.      Interfacing Smart Mobile devices

Mobiles are everywhere. Be it  be business phones like blackberry, Windows, iPhones or tablets/iPads with full blown applications generating and pushing data to central database for analysis, accounting and reporting purpose. Sales force or field agents and inspections generate lot of data with audio and video contents which need to be stored for longer period of process compliance purposes or reference purposes. Nowadays devices comes with inbuilt facilities to store content on local storage or on cloud, it has just become an option and cloud storage just a click away. Because of limited processing power, memory and bandwidth, data from mobile devices need to be pushed /pulled more frequently with availability requirement from anywhere in the world.

Cloud storage proves very efficient option in such cases providing complete data availability; data transfer rate doesn’t matter here since anyways it will happen via Internet with on-premise data center.

3.      Unstructured Data Storage

This is data is largest in size in any organization and uncontrollable also. People create copies of documents and version them as they want, it is difficult to track or control the pattern that users manage documents including emails, text documents, images/photos, manuals, training contents, proposals, marketing contents, accounting statements etc.

As per An IDC paper, “The Diverse and Exploding Digital Universe,” highlights how a single email with 1MB attachment when sent to four people consumes a total of 51MB of storage. (Source: “The Diverse and Exploding Digital Universe, An Updated Forecast of Worldwide Information Growth Through 2011,” March 2008, by International Data Corporation.) In other words, email suffers from attachment size limitations and is also an inefficient way of data sharing.

If constraints are applied on storage sizes, users tend to delete the content which may again create problems in accessing that in future. So to deal with such situations, strong storage policy is needed appreciating business need and impact of data availability for the organization. There should be a flexible way of data sharing which will increase collaboration in the organization along with an approach should prove cost effective and add value in terms of availability, disaster recovery, data redundancies, backups and versioning support.

Cloud storage helps you address all of above concerns fostering effective data storage, sharing, availability and pay as you storage option.

4.      Backup/ Retention/Preservation to Cloud

I have met with one customer who is into construction business from homes to business towers to ships and dams. Company operates in 7 countries following country specific policies and regulations for record keeping. Some of the countries like U.S and Canada need all the records for a construction project to be retained for 10 to 15 years. Data should be recoverable and available when it is required.

One may serve such request by regular maintenance of your storage policies and infrastructure considering backups, verifications, retention, duplication activities. Not only for the regulatory compliances but it is also important for every organization to retain, backup and preserve their data which is a real asset and outcome of thousands of hours of work.

topic 2

Cloud storage takes all of your responsibilities of storing your data consistently for years you want to store. Outsourcing such as pain area will allow your IT to focus more on innovation and value added service for the businesses and not merely on record keeping purpose. Below are some of the instances where cloud storage has been used from some time -

  • Backup for individual machine & laptops to cloud E.g Cirrustore
  • Backup of cloud computing data like OS images Eg: vsphere, Openstack Swift storage
  • Backup for one cloud provider to another – double sure it!
  • Restore the data backed up in cloud- Retention period and Secure deletion purposes
  • Data Preservation (is distinguished from Archive/Retention) in that the goal of preservation is to actively maintain the upkeep of information, most likely for long periods of time. E.g. Preserve Libraries & University Archives / repositories
  • Archive data that is less sensitive to latency

topic 3

5.      Databases in cloud

Databases, may those be SQL or NoSQL can happily reside in the cloud. Some of the Cloud service provides like Microsoft Windows Azure has SQL DB database provided in a ‘Database as a service’ mode. Microsoft has partnered with other database providers like MySQL to make it available on their cloud platform. There is a long list of databases supported on cloud platform like –

  • Cassandra
  • Bigtable
  • Hbase
  • Hypertable
  • Neo4j
  • mongoDB
  • Azure tables

SQL Database can deployed in cloud could be your primary databases, backup copies or as a secondary data source purpose like reporting purpose. SQL database in the cloud are charged based on monthly database size.

NoSQL databases present very cost effective ways for managing data and there are very good example in the industry that how people are using it for storing large data sets in the NoSQL database like Azure tables with partitioning policies spanning data in multiple datacenters around the world.

6.      Cloud Storage for cloud computing

If you are using cloud computing for any reason, there is high possibility that you will fall in love with cloud storage. Most of the cloud providers have storage integrated with other offerings they have.

Whenever we use cloud platform like Microsoft Windows Azure either in IaaS, PaaS or SaaS mode, you are actually using Azure Cloud storage in some or other format.

One of the typical examples when people use Cloud storage is when they use IaaS offerings-

  • VM Image Store :Image of the Guest OS which is made available to hypervisors for staring a VM
  • Guest Auxiliary storage: Provision the storage space, at a given QoS, which the guest needs beyond the boot storage.
  • People store entire deployment packages in cloud storage in PaaS mode for providing autoscaling feature to add more servers in response to traffic attracted by the application.

7.      Global Content Distribution in Cloud

Global content distribution is not a new mechanism to boost application performance when you have wide range of users around the globe and your application content is cached at many places nearer to the users for faster delivery to them. With the advent of Cloud computing, this mechanism has become more powerful with integrated support from the cloud providers like Microsoft has CDN(Content distribution network) feature as of Windows Azure platform, users has to perform few clicks to configure it for their application and your site will be transformed with tremendous boost in content delivery. With increased number of CDN nodes around the globe, latency and scalability issues are being addressed very proactively and easily.

8.      Scientific calculations and researchers to collaborate and discover

Scientific societies and researchers need large storage systems to store their simulations during their research. Not all scientists have liberty to buy storage systems and maintain them. Cloud storage provides efficient technique for them to use actually when required, pay for use and release the resources once results are drawn from the calculations. Cloud storage has been considered good candidate for storing content generated in digital movie production. Computer generated movie production generates huge data which need to be stored for short period of movie production may be for few months.

For e.g. movie like ‘avatar’ generated one petabyte or one million gigabytes of data which was stored using Microsoft digital asset management solution’ which could be today stored in Cloud storage with added benefits.

Hope this helps!  :)

Laxmikant Patil

Why to use Windows Azure Storage against local portable hard disk?


Since the inception of computers, enterprises had been using hard disks for data storage and transfer purpose. Most popular option is ‘portable hard disk’ being cost effective, easy to use and ability to carry anywhere features. Because of success of portable hard disks enterprises got attracted to it and started leveraging them for business data storage, backup and archival purposes, which is not the purpose these disks are meant for. Portable hard disks were developed for storage of temporary data and primarily for portability purpose.

Companies need to look at more reliable storage solution considering below mentioned criteria’s, because data not available on time or loss of data is as bad as data was never available!

Below is the comparison summary between ‘portable hard disk’ and ‘Windows Azure Storage services’ storage options against different criteria’s.

Sr. No

Criteria Portable Hard Disk Cloud(Windows Azure)

Winner

1

Data Security Low – Easily accessible High- Always Secure Access Azure Storage

2

Ease of data Access Data Access is easy as one has to just plug the HDD in USB Internet based data access HDD

3

Portability High Low – but data is available around the world via Internet

HDD

4 Reach Low – Need to physically carry everywhere High – Data accessible globally

Azure Storage

5

Disaster Recovery Low – Very less chances of data recovery High – Inbuilt disaster recovery. Data gets copied at 3 places and   will be automatically made available in disaster recovery scenarios

Azure Storage

6

Data redundancy Low – Need to implement explicitly which costs your more High – Inbuilt data redundancy. Data gets copied at 3 places

Azure Storage

7 Availability Low- Vulnerable to numerous environmental conditions High – 99.9% promised availability with world class data centers   support

Azure Storage

8

Performance High – Local access Low- Internet based access HDD
9 Maintenance Need more care in periodic verifying of device No maintenance needed

Azure Storage

10

Vulnerable to physical damage, heat, dust, wear and tear Yes- highly vulnerable No

Azure Storage

11

Life of storage device 2-3 years max Virtually unlimited Azure Storage
12 Risk of device theft High Low

Azure Storage

13

Data access concurrency Cannot be accessed concurrently for more than few users Can be accessed by large users concurrently

Azure Storage

14

Governed SLAs  No YES – by Microsoft Azure Storage
15 Device Driver needs YES NO

Azure Storage

16

Data Access time Less because of local data transfer More because of internet based data transfer HDD
17 Price 1 TB for Approx. $100 $0.07 per GB per month

HDD

18

Storage Capacity Fixed – need to decide at the time of buying Virtually unlimited Azure Storage
19 Storage flexibility If data size increases, Data storage cannot grow by itself Cloud storage supports scalability out of the box with ability to   storage unlimited data. Flexibility in terms of Pay as you go model

Azure Storage

20

Pricing Model Capex – Capital investment is needed. Opex – Only monthly usage charges need to pay, no upfront commitment Azure Storage
21 Focus Organization need to spend time and give focus on maintaining HDD in   good way along with redundant copies of it No need to spend additional minute in caring about the storage once   data is uploaded.

Azure Storage

Just to conclude, ‘Windows Azure Storage Services’ wins in most of the cases and proves to be best option for storage purpose. Use this information wisely in your scenario to analyze the benefits scenarios.

Hope this helps !

Laxmikant Patil :)

Structured Web: 2025


I. INTRODUCTION

If we look forward to year 2025, where we will have big dreams realized like Nanotechnology, artificial intelligence, next generation cloud and high performance computing. Impact of such technologies on overall human life is unimaginable at this point in time. We are not far away from tiny Nano factories and Nano robots at home doing some smart job for us. Computers around us will be million times faster, smaller, ready for you to serve within fraction of energy consumption as compared to today. These possibilities are beautiful and are likely to be realized but one important question is ‘Are we ready for that?’, ‘Are we putting correct foundation for next generation computing?’ Answer may be a ‘YES’ or ‘NO’ basis individuals perceptions and context. However, it would be certainly ‘No’ if we look at the current un-structured nature of World Wide Web, the biggest information store freely available over our fingertips.

Due to tremendous size of Web, the way we have organized our web resources and the rate of web adoption in developing countries, soon it will become difficult to identify relevant information and services of interest easily. Total dependence on merely text based search engines for information identification will not be sufficient and we will lose credible information which search engines cannot put forward effectively and such a loss may become unaffordable in near future.

Lot of research has been happening around web standardization like research on classifying web sites by Christoph Lindemann[2] and Lars Littig[2], research on extracting and managing structured Web Data by Michael John Cafarella[1] is remarkable.

This paper advises few techniques on structuring the Web to make it best usable.

This is the first paper from the series targeted towards research on ‘Structured Web: 2025’ topic.

structured web 1

Fig. 1  Conceptual view of Web showing scattered information without specific structure

II. PERSPECTIVE

Due to the heterogeneity of the Web and its lack of structure, it is crucial to identify properties of a Web resource that best reflect its functionality.  In Relational Database world, we call it a Schema. If we want to read any tuple from database, we need to first know its schema. This principle is equally applicable to Web resource as well. Once we know the schema, second step is, we should allow database tuple to be read by anybody.

III. APPROACH

Here I propose two step methodology to describe the structure of Web resource.

A. Every Web resource should describe and expose its properties.

B. Every Web resource should be accessible using unified structure.

Here I am considering Web resource as everything which will be publicly accessible.

IV. PROPERTIES DESCRIPTION

This applies to one of the major web resource i.e. Web site. Every Web site should describe its schema using below properties and should expose it for public access.

                                                                                                                                                                           TABLE I
Web site properties

Sr. No. Web Site Properties Description

Category

Element
1

Domain

Domain
2

Presence

Country
3 Languages
4 Time Zone
5 Currency
6

Web Content

Images
7 Text
8 Video
9 Audio
10 XML
11 XHTML
12 RSS
13 Documents (Word, PDF, XLS etc.)
14

Security

Secure
15

Audience

Adult
16

Volume

Size of Pages
17 Count of Pages
18 External site out degree
19

Technical realization

JavaScript, or another scripting
20

Domain dictionary

Domain dictionary keywords
21

Popular URLs

Popular URLs of the site
22

Rank

Rank(1…10)
23

Subdomains

Subdomains
24

Web resource structure

See section V.

Using above information available with each Web site, organizations can write crawlers, which will visit web sites and retrieve these details to maintain database of all this information.

Where -

Domain dictionary keywords can be used by search engines to index the web site against those keywords.

Security signifies if that website can be openly used by anybody or registration is required.

V. RESOURCE ACCESSIBILITY

Once we understand about web site properties, we will be able to understand general structure of it. Next level of categorization is done using how actual web site content is made available for public access. This content access is different from content access using rendered web page. By directly exposing content using URLs will help categorising overall information in terms of relational database table like below.

TABLE II

WEB resource structure

Sr. No. Web site content access structure
Content Type Name URL
1 Image Einstein.png http://www.example.com/Einstein.png
2 Image James Cameron http://www.example.com/JamesC.png
3 PDF USLReport http://www.example.com/USL.pdf
4 Text Football Game http://www.example.com/FootballGame.htm
5 Video President Speech http://www.example.com/Presdspeech.mp4
.. .. .. ..
.. .. .. ..

A. WebResource_Properties.XML file

Now the question is how any web site will expose properties and content access structure to the outside world. Answer is one XML file with standard schema that should be published by every website owner. This file would be WebResource_Properties.XML. This file should be present in each sites root virtual folder and should be accessible publicly by using below URL format –

Http://www.example.com/WebResource_Properties.xml

Using above mechanism, we can build relational database table for all the websites exposing web resource properties.

One can easily write piece of software which will provide you list of all sites from ‘Ireland, in Health care domain, with Audio and images, having page count >20 without any security’ for accessing content.

B. Ranking Website

Another way of classifying web resources/web sites is ranking them. This ranking should be done basis

  • Relevant Content volume and quality
  • No. of users and/or web traffic

Web site ranking should be done by independent organizations to provide real usability aspect to the world. Rank is always linked to Domain, so while comparing ranks domain always comes into picture.

C. Domain

Some of the domains can be listed as Affiliate site, Archive Site, Blogs, Corporate site, Commerce Site, database site, development site, directory site, download site, employment site etc.

VI. TWO VIEWS OF WEB SITE

Figure below shows two views of web site as –

A. View which is rendered in the browser and user can see it directly. Search engines works on this view for performing indexing on web site. Search engine cannot reach to the web resource which has got no link in the browser rendered page. Search engine cannot crawl the web sites which have got some files on web servers without links provided in web pages.

B. Second view is the view provided through WebResources_Properties xml file, sample as shown in right side of this figure.

strcutured web 2

Fig. 2  Web Virtual directory and two views of it

VII.  CONCLUSION

By implementing above guidelines, web information can be structured to some level which allows us to leverage following advantages.

A. Technology neutral way of categorizing of web sites

Using above method web sites can be categorized and web can be structured in a technology neutral way.

B. Improved search engine optimization

Now search engines need not just depend on text based indexing, additional web resource properties can help in getting meaningful search results.

C. Minimal work to get started

Web resource owners don’t need to make any changes in their web applications. Just one XML file will help bring in lots of difference.

Figure below shows conceptual view of Web when such structuring will happen over a period. Web being a massive data store, it will take time for people to adopt such standards and apply them.

strcutured web 3

Fig 3. – Conceptual view of Web showing structured information after employing above techniques

Important point is if we don’t take action on time we will be at great loss where millions of ideas/research/opinions by billions of people might get into dark ages just because nobody could find it at correct time and carry on further work. People will keep on reinventing the wheel, and next generation will blame on us because we could not manage the Web with great responsibly. If we start today, hope is entire Web will be structured data source by 2025 and next generation might use structured query language to search the Web seamlessly.

Because “Information could not be found easily is as good as information is not present.”

VIII. REFERENCES

[1]        Michael John Cafarella, Extracting and Managing Structured Web Data, university of Washington, 2009

[2]        Chrisoph Lindemann and Lars Littig, Classifying web sites, University of Leipzig, Johannisgasse 26, 2007

[3]        John M. Pierre, On the Automated Classification of Web Sites, California UAS, 2001

Follow

Get every new post delivered to your Inbox.

Join 204 other followers