Thursday, 24 January 2008
Hu Yoshida, CTO, Hitachi Data Systems
1. Controlling Carbon Emissions: With increasing concern about global warming we will see more governments impose guidelines and legislation around carbon emissions. Major corporations will set targets for reduction of carbon emissions.
A major source of carbon emissions comes from the generation of electricity. The increasing demand for computer power, network bandwidth, and storage capacity will increase the need for data centre power and cooling.The US government just completed a study which estimated that the IT sector consumed about 61 billion kilowatt-hours (kWh) in 2006 (1.5 percent of total U.S. electricity consumption) for a total electricity cost of about $4.5 billion. This is expected to double in the next 5 years. Some cities like London and New York are running out of electrical capacity and data centres are forced to relocate to other areas where power is available. This will require facility upgrades and investment in Green technology.
In September of 2007 Hitachi ltd, announced a program in Japan that is known as CoolCentre50 which is targeted at reducing power consumption in their Yokohama and Okayama data centre by 50% in 5 years. This effort encompasses all of the groups in Hitachi including, air conditioning, power generation, IT equipment and management software. In support of this goal, Naoya Takahashi, the Executive General Manager and Vice President of the Hitachi’s Information and Telecommunications Group announced the Harmonious Green Plan which aims to reduce 330,000 tons of CO2 in the next 5 years through the development of power saving IT products like storage and server virtualisation.
2. Economic Uncertainty: The collapse of the housing market in the US, high oil prices and the falling dollar will create economic uncertainty. Budgets will be tight and IT will have to do more with less. Doing more with less will drive It to consider ways to consolidate IT resources through virtualisation, increase utilisation of resources such as server cycles and storage capacities, eliminate redundancies where ever possible through de-duplication and single instance store, and reduce the working set of production data through the aggressive use of archive products.
3. Increasing Use of Archiving: Structured data like databases are exploding as they are required to hold more data, longer, for compliance reasons. Semi structured data, like email, web pages, and document management data are increasing dramatically. Corporate email quotas will increase from less than 200 MB to 2GB in order to support new knowledge workers and compete with free mail box offerings from Google and AOL. An avalanche of unstructured data will be driven by RFID tags, Smart cards, and sensors that monitor everything from heartbeats to border crossings. The new Airbus and Boeing Dream Liners will generate terabytes (TBs) of data on each flight. All these pressures will drive the need to archive data in order to reduce the working set of production data. This will call for new types of archiving systems that can scale to petabytes and provide the ability to search for content across different modalities of data. Creating a separate archive for each type of data will not help the problem.
4. Awareness of Storage Deficiencies: There will be a growing awareness that the storage of data has become highly inefficient, with low utilisation, stranded storage, too many redundant copies, low access speeds, inefficient search, and disruptive movement and migration. Continuing to buy more of the same old storage architectures will no longer be an option. Buying faster storage processors with larger capacity disks on the same 20 year old architectures will no longer be viable. New storage architectures will be required to meet these changing demands. A new architecture that can scale performance, connectivity, and capacity, non-disruptively to multiple petabytes is required. It must also be able to provide new data and storage services like multi-protocol ingestion and common search, across heterogeneous storage arrays with centralised management and secure protection.
5. Data Mobility will be a Key Requirement: With the need for continuous application availability, IT will need the ability to move data without disruption to the application. While software data movers have been used in the past, they steal processor cycles from the application and are limited to slow speed IP links to move data. As the volume of data increases, this becomes too disruptive. The movement of data will have to be offloaded to a storage system which can move data over high speed Fibre Channel links without the need for the application’s processor cycles. This will be increasingly more important for migration of data during storage upgrades to larger and larger storage capacity frames.
6. Control Unit Virtualisation of Storage: Control Unit Virtualisation of Storage will be recognised as the only approach to storage virtualisation that can add value to existing storage arrays. Industry analysts like Dr. Kevin McIsaac, of Intelligent Business Research Services Pty in Australia points out ”The idea of being able to layer (network-based) virtualisation over existing storage arrays is seriously flawed”. He points out that this “results in a lowest common denominator view of the infrastructure, eliminating the value-added features of the array.” This type of virtualisation adds another layer of complexity introduces a performance bottleneck, becomes another potential source of failure, and a vendor lock-in. A control unit based approach to virtualisation is able to leverage all the rich functionality of the control unit to enhance the functionality of lower cost or legacy tiers of storage arrays. A control unit based approach to virtualisation will enable less capable storage systems to utilise the value added services in that control unit like data mobility functions or thin provisioning capabilities.
7. Services Oriented Storage: Services Oriented Storage will become a requisite complement to Services Oriented Architecture (SOA) in the application space and to Services Oriented Infrastructure in the infrastructure space in order to achieve the dynamic data centre of the future. SOA depends on a virtualisation layer provided by XML which enables applications to share information and utilise common services like billing. Services Oriented Infrastructure depends on a virtualisation layer provided by products like VMWare which enables operating systems to share the resources of a processor platform. Services Oriented Storage requires a virtualisation layer in the storage control unit which enables other storage systems to leverage its services like a high performance global cache, distance replication, tiered storage, and thin provisioning.
8. Convergence of Content, File, and Block based Storage Services: Instead of separate stove pipe storage systems for content (archive), file and block storage system, we will see the convergence of these storage types to a common virtualisation platform. High availability clusters of content servers and file servers will use a common block virtualisation services platform, under one common set of management tools. This will enable content servers or file servers to leverage common block services like distance replication, thin provisioning, or virtualisation of heterogeneous storage systems
9. Thin Provisioning: Thin provisioning will provide the biggest benefit in increasing the utilisation of storage by eliminating the waste of allocated but unused storage capacity. This savings is multiplied many times over by eliminating the need to copy that allocated but unused capacity every time a copy is required for backup cycles, replication, data mining, development test, data distribution, etc. The implementation of thin provisioning should be provided as a service on a storage virtualisation platform so that it can benefit existing storage systems through virtualisation. The benefits of thin provisioning would be defeated if it requires yet another stand alone storage system. This ability to increase utilisation will be embraced by Green advocate and will be seen as a way to contain costs.
10. Deduplication: Deduplication will be implemented by all the major backup vendors. Deduplication is especially effective in eliminating duplicated data in backups. The ability to reduce a stream of data by 20 to 30 times will be extremely valuable in reducing the cost of storing data to the point that it will be feasible to store backup data to disk rather than to tape where the operational, availability, and reliability characteristics are better. Other forms of deduplication like single instance store for archives, and copy on write for snapshots will become more prevalent.
Showing posts with label Storage. Show all posts
Showing posts with label Storage. Show all posts
Feb 21, 2008
IDC's 2008 Storage Predictions
IDC recently released their 2008 Top 10 Storage Predictions (Doc #209796, Worldwide Storage 2008 Top 10 Predictions: New Paradigms.) In my previous post I listed our Top 10 predictions, let's look at IDC's take...
IDC's 2008 Storage Predictions:
1. Storage as a Service (SaaS): See our Trend #10, looks like this may be the year for SaaS. The best thing that SaaS does is this: It lowers the barrier of entry for start ups. Want a perfect example? SmugMug.com stores approx. 205,000,000 photos for its customers using Amazon S3 and estimates they will save around $2M in storage costs just this year.
2. Ro le-based Storage Systems: I had to read deeper into the report to see what IDC is talking about here. Their summary describes it as storage solutions that will target a specific issue (backup/restore, archive, etc.) and one that will leverage "standard server architectures" and add value through "advanced software." Hmmm, sounds an awful lot like Sun's Open Storage Platform (see trend #1) - Standard server architecture: check. Advanced software features: ZFS? - check. Role-based: well, add VTL software to Sun's X4500 and you have a backup/restore device, add IPconfigure software and you have a Digital Video Recorder (DVR), add Greenplum software and you have a Data Warehouse application - check!
3. Object -based Storage Systems: See our trend #7. IDC makes an excellent distinction in their report that I must highlight...a lot of storage applications classify data (moving it to the appropriate storage tier) by looking at data workloads. But several different types of workloads can be used by a single application. The type of data + the application it is tied to may be a better way to classify it.
4. Solid-State Disk (SSD): See our trend #9. IDC states that SSD will enter into the data center as storage tier 0 and I agree. Flash promoters and disk vendors are already starting to pull up benchmarks against each other, but an integrated approach will probably prevail. For a fun SSD RAID implementation see the "Battleship Mtron." StorageMojo has a balanced take on SSD.
5. Virtual Servers (e.g. VMware) will be killer app for iSCSI: IDC predicts that just under half of virtualized servers connected to SANs will be connected to iSCSI. They state that manufactures selling iSCSI systems will find the servers they are connected to will be running server virtualization. (As virtual server admins are more comfortable with IP & Ethernet technology).
6. Value-Added Storage Services Will Begin to Be Divorced from Storage Subsystems, Resulting in Further Commoditization of Storage Subsystems: Quite a mouthful - but if you want an example, check out Sun's ZFS and its point-in-time-copy, volume management, administration, Copy-on-write and RAID Storage Services!
7. Disk Encryption: See our trend #5. We did it for tape, disk is a logical next step. Key Management continues to be the key part of encryption solutions, whether they are tape or disk based...
8. Vendors Will Create More Attractive "All In One" Solutions Using an Integrated Storage and Server Approach: Look no further than the SunFire X4500 (aka "Thumper")
9. "Green" Initiatives Will Spark Some Hardware Refreshes: IDC predicts that Eco messaging will evolve to vendors helping users connect business practices to green storage, as well as IT focusing on how to address the environmental impacts of manufacturing and disposing of older storage equipment.
10. De-Dup, VTL & Thin Provisioning become standard options: See our #3 and #4 trends. What were emerging technologies in 2007, will become "rights to play" in 2008 …
IDC's 2008 Storage Predictions:
1. Storage as a Service (SaaS): See our Trend #10, looks like this may be the year for SaaS. The best thing that SaaS does is this: It lowers the barrier of entry for start ups. Want a perfect example? SmugMug.com stores approx. 205,000,000 photos for its customers using Amazon S3 and estimates they will save around $2M in storage costs just this year.
2. Ro le-based Storage Systems: I had to read deeper into the report to see what IDC is talking about here. Their summary describes it as storage solutions that will target a specific issue (backup/restore, archive, etc.) and one that will leverage "standard server architectures" and add value through "advanced software." Hmmm, sounds an awful lot like Sun's Open Storage Platform (see trend #1) - Standard server architecture: check. Advanced software features: ZFS? - check. Role-based: well, add VTL software to Sun's X4500 and you have a backup/restore device, add IPconfigure software and you have a Digital Video Recorder (DVR), add Greenplum software and you have a Data Warehouse application - check!
3. Object -based Storage Systems: See our trend #7. IDC makes an excellent distinction in their report that I must highlight...a lot of storage applications classify data (moving it to the appropriate storage tier) by looking at data workloads. But several different types of workloads can be used by a single application. The type of data + the application it is tied to may be a better way to classify it.
4. Solid-State Disk (SSD): See our trend #9. IDC states that SSD will enter into the data center as storage tier 0 and I agree. Flash promoters and disk vendors are already starting to pull up benchmarks against each other, but an integrated approach will probably prevail. For a fun SSD RAID implementation see the "Battleship Mtron." StorageMojo has a balanced take on SSD.
5. Virtual Servers (e.g. VMware) will be killer app for iSCSI: IDC predicts that just under half of virtualized servers connected to SANs will be connected to iSCSI. They state that manufactures selling iSCSI systems will find the servers they are connected to will be running server virtualization. (As virtual server admins are more comfortable with IP & Ethernet technology).
6. Value-Added Storage Services Will Begin to Be Divorced from Storage Subsystems, Resulting in Further Commoditization of Storage Subsystems: Quite a mouthful - but if you want an example, check out Sun's ZFS and its point-in-time-copy, volume management, administration, Copy-on-write and RAID Storage Services!
7. Disk Encryption: See our trend #5. We did it for tape, disk is a logical next step. Key Management continues to be the key part of encryption solutions, whether they are tape or disk based...
8. Vendors Will Create More Attractive "All In One" Solutions Using an Integrated Storage and Server Approach: Look no further than the SunFire X4500 (aka "Thumper")
9. "Green" Initiatives Will Spark Some Hardware Refreshes: IDC predicts that Eco messaging will evolve to vendors helping users connect business practices to green storage, as well as IT focusing on how to address the environmental impacts of manufacturing and disposing of older storage equipment.
10. De-Dup, VTL & Thin Provisioning become standard options: See our #3 and #4 trends. What were emerging technologies in 2007, will become "rights to play" in 2008 …
Top 10 Storage Technology Trends
Source Friday Nov 30, 2007
It’s getting to be the end of the year – and Sun, like every other vendor I assume, is looking at the new technology trends that have (or will) impact our industry.
My team and I were asked to evaluate some – to see where to prioritize future investments, evaluate current and future competitive threats, etc. Ensuring we stay ahead of the technology game. So below is our very own top 10 list of emerging technologies. (As well as some color to what Sun is doing here)
I’m sure we’re missing some and comments and claims are our own, they don’t represent any commitment or position from Sun Corporate (For official news on Sun go here…)
Top 10 Storage Technology Trends:
1. Open Storage Platform (aka general purpose storage, open source storage): Trend #1 is a term we coined, so it may not sound familiar. It is a combination of market trends as well as a direction Sun is taking with its newer products. The concept of a common platform is not new – several vendors have tried to build one platform that can run multiple storage applications, saving users time and money. “Open” is a relatively new concept for storage, but not for software or servers. There are generally three components that make up an Open Storage Platform:
. General Purpose Components: General-purpose servers, processors, storage and operating systems are now being deployed in enterprise-class storage devices. Previously, storage vendors had built proprietary operating systems, ASIC chips and other custom-built components. As commodity chips, components and software have matured, storage vendors are now using general purpose components in their systems. The cost savings are significant - however, most vendors' prices remain the same (contributing to vendor margin). Sun's philosophy is to pass these savings to the consumer. Enterprise-class systems based on general purpose components give customers higher-value systems at a fraction of the cost (see graphic below). They are also much more flexible, as they can be re-purposed for other uses.
. OS/File System Storage Services: Traditional appliances charge customers extra in software licensing costs for data management services like administration, replication or volume management. What if this functionality came already embedded in the storage system itself? Sun's newest file system, ZFS, has started to incorporate these services at the File System level. ZFS deploys point-in-time-copy, volume management, administration and data integrity features like Copy-on-write and RAID. Storage services deployed at the OS-level have several benefits, including efficiency, performance, reliability and affordability (no more licenses for extra software needed). And before we think this is a new concept, mainframe has been doing it for years . The new concept is doing this in the open systems space – and Sun Solaris ZFS is leading the charge.
. Open Source Storage Software: The ability to download software, test it and add features to it is critical to developers building new applications – we’ve seen this in the OS market. But what about storage? This has been an investment area of Sun’s - Sun actually offers one of the most complete open source storage software stacks (from protocols to drivers to data management software). Developers can build their own storage solutions and sell them by leveraging Sun’s open source software (see the Nexenta Storage Appliance for a perfect example). Open source has an added benefit to customers with in-house development resources - customers can deploy new software features by searching for it in the open source community or even developing it themselves (why wait for vendor roadmaps?). This is not possible with a proprietary appliance.
The simplest way to show the impact the Open Storage Platform concept will have on the storage industry is a basic economic comparison. We used IDC's Pricing Database to compare over 50 actual purchase orders of enterprise disk (Sun ST9900, EMC DMC3), Midrange Disk (Sun ST6140, EMC CX3) and a Sun product built on our Open Storage Platform (the SunFire X4500):
- Enterprise Disk = $18.84/GB
- Midrange Disk = $10.39/GB
- Open Storage Platform = $1.50/GB
(7x less than Midrange and 13x less than Enterprise )
NOTE: There are certain applications and features that must run on higher-end disk. But at a $1.50, customers will be compelled to find which applications should be running on the SunFire X4500. What is even more remarkable, the SunFire X4500 includes the server, OS, data services, storage and networking components - all for $1.50/GB!
Game-changing economics…
2. Virtualization : Where to begin? The benefits are obvious – massive savings through optimization, consolidation and optimization. The best definition of virtualization I have seen came from the 4 51 Group’s Virtualization Report which defined Virtualization as, “A software abstraction layer that permits aggregation, emulation or partitioning.” Let’s look at the different types of virtualization out there today
. Server Virtualization: The ability to host Windows, Linux and Solaris operating systems on one platform has literally taken the market by storm. A huge boon to IT managers and developers alike. VMWare led the charge with some start-ups and open source initiatives in tow. On October 5, 2007 Sun entered the race with Sun xVM. (And if virtualization has hit the systems world, expect to see it hit the desktop market soon…)
. File Virtualization (a.k.a Clustering, Global Namespace, Unified Namespace, NAS Virtualization): A few years ago we were talking about Grid Storage – a bunch of storage nodes acting as one, single system. File virtualization is shaping up to be the technology that will take us there. In 2005 & 2006 we saw EMC buy Rainfinity, NetApp buy Spinnaker and Brocade buy NuView. In recent years we we have seen EMC announce Rainfinity, NetApp announce a rough start to Data OnTap GX (based on Spinnaker technology) and file virtualization start ups Ibrix, Isilon and LeftHand Networks continue to grow. Expect this market to heat up, and consolidate as start ups are either bought, or simply run out of runway.
Virtual Tape: A disk system that emulates a tape library. Virtual Tape has been in the mainframe market for years, and Sun is the market leader here. Virtual Tape Libraries (VTL) in the open systems market are relatively new – disk prices have eroded to a point where backup administrators can now take advantage of disk’s access speeds without replacing their exiting backup infrastructure. IDC pegs this market as small, but growing at 16.2% (2006-2011 CAGR).
. Virtual Disk (aka Storage Virtualization): A disk system that aggregates 3rd-party disk and offers disk partitioning. A necessity for anyone that needs to consolidate their environment and/or make data migration between independent systems easier. Virtual Disk Systems include IBM’s San Volume Controller (SVC), FalconStor’s IPStor and Sun’s award-winning StorageTek 9000 arrays.
3. Thin Provisioning: Better system utilization is the name of the game. Most admins know that the utilization rates on their disk systems ar e not where they need to be. Thin Provisioning allows admins to allocate or provision space to specific applications, making full use of their system’s capacity. 3PAR spearheaded open systems Thin Provisioning and NetApp offers it as a part of Data OnTap. Sun announced Thin Provisioning on its StorageTek 9990V system in May – meaning consumers can have the world’s fastest enterprise array, Virtual Disk AND Thin Provisioning all on one platform.
Pretty cool…
4. Data Deduplication (aka De-dup, Single-instance storage): In a world where there is more data coming into a company than can possibly be managed – data compression ratios ranging from 10:1 to 50:1 sound pretty darn nice ( See how De-dup works here). Data Domain, Diligent, FalconStor and other upstarts get credit for bringing this new technology to market and larger vendors are quickly following suite. De-dup is still emerging, can have performance issues and does not work perfectly for every application – but economics dictate its worth consumers investigating where it can work for them.
There are two emerging de-dup architectures: “Inline” – where the de-dup magic happens in real-time, as data comes into the system, as found in Diligent's ProtecTIER appliance. Or “Post-Processing” where the magic happens as a secondary process after the backup job, as found in FalconStor’s Single Instance Repository (SIR) software. Both have their pros and cons, and deciding which approach to use depends on balancing your performance vs. complexity needs. For the record, Sun sells both….
5. Data Encryption: One need only read the horror stories of lost tape and disk drives to see the importance of data encryption. While it has been around for a while – the need has never been greater. Growing storage capacity has caused another problem – one can store a lot of personnel records on a single cartridge or drive. In an age of identify theft, losing one storage device can put a company out of business. The new trend is not how to encrypt, but where to encrypt… On the host server? On an appliance in the network? In the storage device itself? Decru (since bought by NetApp) benefited from this trend with their encryption appliance.
I once worked with a brilliant engineer whose favorite saying was “never put a product where a feature should be. ” I’ d say this was Sun’s philosophy when we delivered the Sun StorageTek T10000 tape drive. Put simply, Sun put an encryption chip next to the compression chip on the drive – so data is encrypted as it is fed onto the tape. Simple and affordable – no extra appliance needed. Sun also offers the StorageTek Crypto Key Management Station to centrally authorize, secure and manage encryption keys.
6. Eco Storage (aka Green Storage/IT): I freely admit that when I was first approached with “Green Storage” I was a skeptic. I would have also never guessed Al Gore would win the Nobel Peace Prize! But Eco also stands for Economics. If you save power and footprint, and the world while you are at it – who can argue with that? But the challenge for storage customers will be sorting through the vendors who make REAL Eco investments vs. the ones that just add “Eco” or “Green” to their marketing collateral. Sun’s in the “real” category, investing heavily in Eco IT. Sun’s Eco efforts can be seen here...
7. Object Archive (aka CAS, Application Aware Archive): The dizzying array of regulations, compliance requirements and influx of data have made the archive market one of the fastest growing markets in IT and stor age. And customers must continually evaluate which archive approach will work best for them. The trend here is to “build a better mousetrap archive.” The challenge is this, an archive system must:
- Store a lot of data affordably
- Have WORM functionality so documents show up un-altered in an audit or court of law
- Be easily and quickly accessible
- For more than 100 years…
Easier said than done. While tape continues to be the old staple in archive, and faster and denser tape systems are coming out each year – a lot of new innovations are happening on the disk and software side. Disk-based object-level archive systems include EMC’s Centera, HP’s RISS and Sun’s StorageTek 5800 “Honeycomb” pictured at right. Sun’s ST5800 system uses advanced meta data features and processors close to each storage cluster for fast access to deep archives – great for digital library, Web 2.0 and HPC applications and environments.
But do keep in mind for deep archive; Sun’s StorageTek SL8500 Tape Library is tough to beat – just one library's max raw capacity is 56 Petabytes, and data sitting on tape consumes 0 kilowatts and generates 0 CO2… (see above trend #6)
8. New Interfaces, Protocols & Configurations: There is a lot of change happening in storage systems and how they are configured. The three primary ways storage is attached is Direct Attached Storage ( DAS ), Storage Area Networks (SAN) and Network Attached Storage (NAS). A disk system can also be configured in a couple different ways. RAID configurations stripe data across multiple drives and impact a system’s reliability and performance. JBOD (Just a Bunch of Disks) is more affordable because it does not require a disk controller, but provides no data redundancy. New interfaces and protocols will impact each of these markets significantly.
. SAS: In DAS, Serial Attached SCSI (SAS) is a newer serial communication protocol that will make data transfer speeds much faster and at a lower cost. SAS drives have replaced parallel SCSI for internal storage, and SAS HBA external host interfaces are just starting to ship in volume. SAS disk arrays started shipping this year and SAS JBOD are starting to ship. To complement the SAS HBA's and SAS arrays, there are new network components called expanders which are basically switches. Generally SAS disk will be cheaper than Fibre Channel (FC) with similar performance and capacity. (Sun has SAS host I/F, disk I/F and SAS disk drives)
. iSCSI: In the SAN market, the iSCSI protocol is making a significant impact by taking the cost out of expensive Fibre Channel SANs. iSCSI allows users to send SCSI commands over their existing IP networks.
. File Virtualization: In the NAS market, file virtualization is making NAS farms much more scalable and manageable (See trend #2)
. Clustered RAID: A new innovation to watch in the RAID market is horizontally scalable RAID or clustered RAID for large applications. Digi-Data is a small storage company pushing this innovation.
. JBOD: Keep your eye on Sun in the JBOD market. JBOD is more affordable than RAID, but does not have RAID’s redundancy and reliability features. But what if you had an infinitely scalable file system with data integrity and RAID features running on JBOD, say, something like ZFS?
. Unified Storage: Another trend in this area are unified or hybrid devices – storage systems that can handle multiple protocols and interfaces, including iSCSI, Fibre Channel, and NAS – all in one unit. Makes sense to users with a dizzying array of choices in the market.
. 10GbE: Lastly, 10 Gigabit Ethernet or 10GbE is the latest and fastest of the Ethernet standards that will re-shape data center networking – offering a fast, common and affordable network technology for IT and Web 2.0 applications from supercomputing to networked storage.
9. Solid State Disk (aka SSD, Flash, Memory): We see the perfect storm happening around SSD. SSD has no moving parts, making it one of the most reliable and fastest storage mediums in the world. But it is incredibly expensive when you compare $/GB vs. Disk and Tape storage. However, SSD has made a name for itself in the consumer market (digital cameras, phones, iPods, etc.) The price pressure in the consumer arena is enormous – and this had been rapidly eroding the price of SSD. As SSD prices drop, expect to see hybrid disk drives and storage systems that leverage more SSD for greater speed and IOPS.
10. Storage as a Service:
Storage as a service offered over the Internet has been talked about for years – but poor performance and implementations have cooled this trend. However, Amazon has given Storage as a Service a power boost with its Amazon Simple Storage Service (Amazon S3). By leveraging Amazon’s existing e-commerce and storage infrastructure, the company is offering customers storage capacity for $0.15 per GB-Month of storage used – possibly the cheapest $/GB on the planet. And while this may have more play in the consumer market, Amazon could re-invigorate the storage as a service trend. Also keep an eye on Sun’s Internet Service offerings over Network.com … Continue Reading »
It’s getting to be the end of the year – and Sun, like every other vendor I assume, is looking at the new technology trends that have (or will) impact our industry.
My team and I were asked to evaluate some – to see where to prioritize future investments, evaluate current and future competitive threats, etc. Ensuring we stay ahead of the technology game. So below is our very own top 10 list of emerging technologies. (As well as some color to what Sun is doing here)
I’m sure we’re missing some and comments and claims are our own, they don’t represent any commitment or position from Sun Corporate (For official news on Sun go here…)
Top 10 Storage Technology Trends:
1. Open Storage Platform (aka general purpose storage, open source storage): Trend #1 is a term we coined, so it may not sound familiar. It is a combination of market trends as well as a direction Sun is taking with its newer products. The concept of a common platform is not new – several vendors have tried to build one platform that can run multiple storage applications, saving users time and money. “Open” is a relatively new concept for storage, but not for software or servers. There are generally three components that make up an Open Storage Platform:
. General Purpose Components: General-purpose servers, processors, storage and operating systems are now being deployed in enterprise-class storage devices. Previously, storage vendors had built proprietary operating systems, ASIC chips and other custom-built components. As commodity chips, components and software have matured, storage vendors are now using general purpose components in their systems. The cost savings are significant - however, most vendors' prices remain the same (contributing to vendor margin). Sun's philosophy is to pass these savings to the consumer. Enterprise-class systems based on general purpose components give customers higher-value systems at a fraction of the cost (see graphic below). They are also much more flexible, as they can be re-purposed for other uses.
. OS/File System Storage Services: Traditional appliances charge customers extra in software licensing costs for data management services like administration, replication or volume management. What if this functionality came already embedded in the storage system itself? Sun's newest file system, ZFS, has started to incorporate these services at the File System level. ZFS deploys point-in-time-copy, volume management, administration and data integrity features like Copy-on-write and RAID. Storage services deployed at the OS-level have several benefits, including efficiency, performance, reliability and affordability (no more licenses for extra software needed). And before we think this is a new concept, mainframe has been doing it for years . The new concept is doing this in the open systems space – and Sun Solaris ZFS is leading the charge.
. Open Source Storage Software: The ability to download software, test it and add features to it is critical to developers building new applications – we’ve seen this in the OS market. But what about storage? This has been an investment area of Sun’s - Sun actually offers one of the most complete open source storage software stacks (from protocols to drivers to data management software). Developers can build their own storage solutions and sell them by leveraging Sun’s open source software (see the Nexenta Storage Appliance for a perfect example). Open source has an added benefit to customers with in-house development resources - customers can deploy new software features by searching for it in the open source community or even developing it themselves (why wait for vendor roadmaps?). This is not possible with a proprietary appliance.
The simplest way to show the impact the Open Storage Platform concept will have on the storage industry is a basic economic comparison. We used IDC's Pricing Database to compare over 50 actual purchase orders of enterprise disk (Sun ST9900, EMC DMC3), Midrange Disk (Sun ST6140, EMC CX3) and a Sun product built on our Open Storage Platform (the SunFire X4500):
- Enterprise Disk = $18.84/GB
- Midrange Disk = $10.39/GB
- Open Storage Platform = $1.50/GB
(7x less than Midrange and 13x less than Enterprise )
NOTE: There are certain applications and features that must run on higher-end disk. But at a $1.50, customers will be compelled to find which applications should be running on the SunFire X4500. What is even more remarkable, the SunFire X4500 includes the server, OS, data services, storage and networking components - all for $1.50/GB!
Game-changing economics…
2. Virtualization : Where to begin? The benefits are obvious – massive savings through optimization, consolidation and optimization. The best definition of virtualization I have seen came from the 4 51 Group’s Virtualization Report which defined Virtualization as, “A software abstraction layer that permits aggregation, emulation or partitioning.” Let’s look at the different types of virtualization out there today
. Server Virtualization: The ability to host Windows, Linux and Solaris operating systems on one platform has literally taken the market by storm. A huge boon to IT managers and developers alike. VMWare led the charge with some start-ups and open source initiatives in tow. On October 5, 2007 Sun entered the race with Sun xVM. (And if virtualization has hit the systems world, expect to see it hit the desktop market soon…)
. File Virtualization (a.k.a Clustering, Global Namespace, Unified Namespace, NAS Virtualization): A few years ago we were talking about Grid Storage – a bunch of storage nodes acting as one, single system. File virtualization is shaping up to be the technology that will take us there. In 2005 & 2006 we saw EMC buy Rainfinity, NetApp buy Spinnaker and Brocade buy NuView. In recent years we we have seen EMC announce Rainfinity, NetApp announce a rough start to Data OnTap GX (based on Spinnaker technology) and file virtualization start ups Ibrix, Isilon and LeftHand Networks continue to grow. Expect this market to heat up, and consolidate as start ups are either bought, or simply run out of runway.
Virtual Tape: A disk system that emulates a tape library. Virtual Tape has been in the mainframe market for years, and Sun is the market leader here. Virtual Tape Libraries (VTL) in the open systems market are relatively new – disk prices have eroded to a point where backup administrators can now take advantage of disk’s access speeds without replacing their exiting backup infrastructure. IDC pegs this market as small, but growing at 16.2% (2006-2011 CAGR).
. Virtual Disk (aka Storage Virtualization): A disk system that aggregates 3rd-party disk and offers disk partitioning. A necessity for anyone that needs to consolidate their environment and/or make data migration between independent systems easier. Virtual Disk Systems include IBM’s San Volume Controller (SVC), FalconStor’s IPStor and Sun’s award-winning StorageTek 9000 arrays.
3. Thin Provisioning: Better system utilization is the name of the game. Most admins know that the utilization rates on their disk systems ar e not where they need to be. Thin Provisioning allows admins to allocate or provision space to specific applications, making full use of their system’s capacity. 3PAR spearheaded open systems Thin Provisioning and NetApp offers it as a part of Data OnTap. Sun announced Thin Provisioning on its StorageTek 9990V system in May – meaning consumers can have the world’s fastest enterprise array, Virtual Disk AND Thin Provisioning all on one platform.
Pretty cool…
4. Data Deduplication (aka De-dup, Single-instance storage): In a world where there is more data coming into a company than can possibly be managed – data compression ratios ranging from 10:1 to 50:1 sound pretty darn nice ( See how De-dup works here). Data Domain, Diligent, FalconStor and other upstarts get credit for bringing this new technology to market and larger vendors are quickly following suite. De-dup is still emerging, can have performance issues and does not work perfectly for every application – but economics dictate its worth consumers investigating where it can work for them.
There are two emerging de-dup architectures: “Inline” – where the de-dup magic happens in real-time, as data comes into the system, as found in Diligent's ProtecTIER appliance. Or “Post-Processing” where the magic happens as a secondary process after the backup job, as found in FalconStor’s Single Instance Repository (SIR) software. Both have their pros and cons, and deciding which approach to use depends on balancing your performance vs. complexity needs. For the record, Sun sells both….
5. Data Encryption: One need only read the horror stories of lost tape and disk drives to see the importance of data encryption. While it has been around for a while – the need has never been greater. Growing storage capacity has caused another problem – one can store a lot of personnel records on a single cartridge or drive. In an age of identify theft, losing one storage device can put a company out of business. The new trend is not how to encrypt, but where to encrypt… On the host server? On an appliance in the network? In the storage device itself? Decru (since bought by NetApp) benefited from this trend with their encryption appliance.
I once worked with a brilliant engineer whose favorite saying was “never put a product where a feature should be. ” I’ d say this was Sun’s philosophy when we delivered the Sun StorageTek T10000 tape drive. Put simply, Sun put an encryption chip next to the compression chip on the drive – so data is encrypted as it is fed onto the tape. Simple and affordable – no extra appliance needed. Sun also offers the StorageTek Crypto Key Management Station to centrally authorize, secure and manage encryption keys.
6. Eco Storage (aka Green Storage/IT): I freely admit that when I was first approached with “Green Storage” I was a skeptic. I would have also never guessed Al Gore would win the Nobel Peace Prize! But Eco also stands for Economics. If you save power and footprint, and the world while you are at it – who can argue with that? But the challenge for storage customers will be sorting through the vendors who make REAL Eco investments vs. the ones that just add “Eco” or “Green” to their marketing collateral. Sun’s in the “real” category, investing heavily in Eco IT. Sun’s Eco efforts can be seen here...
7. Object Archive (aka CAS, Application Aware Archive): The dizzying array of regulations, compliance requirements and influx of data have made the archive market one of the fastest growing markets in IT and stor age. And customers must continually evaluate which archive approach will work best for them. The trend here is to “build a better mousetrap archive.” The challenge is this, an archive system must:
- Store a lot of data affordably
- Have WORM functionality so documents show up un-altered in an audit or court of law
- Be easily and quickly accessible
- For more than 100 years…
Easier said than done. While tape continues to be the old staple in archive, and faster and denser tape systems are coming out each year – a lot of new innovations are happening on the disk and software side. Disk-based object-level archive systems include EMC’s Centera, HP’s RISS and Sun’s StorageTek 5800 “Honeycomb” pictured at right. Sun’s ST5800 system uses advanced meta data features and processors close to each storage cluster for fast access to deep archives – great for digital library, Web 2.0 and HPC applications and environments.
But do keep in mind for deep archive; Sun’s StorageTek SL8500 Tape Library is tough to beat – just one library's max raw capacity is 56 Petabytes, and data sitting on tape consumes 0 kilowatts and generates 0 CO2… (see above trend #6)
8. New Interfaces, Protocols & Configurations: There is a lot of change happening in storage systems and how they are configured. The three primary ways storage is attached is Direct Attached Storage ( DAS ), Storage Area Networks (SAN) and Network Attached Storage (NAS). A disk system can also be configured in a couple different ways. RAID configurations stripe data across multiple drives and impact a system’s reliability and performance. JBOD (Just a Bunch of Disks) is more affordable because it does not require a disk controller, but provides no data redundancy. New interfaces and protocols will impact each of these markets significantly.
. SAS: In DAS, Serial Attached SCSI (SAS) is a newer serial communication protocol that will make data transfer speeds much faster and at a lower cost. SAS drives have replaced parallel SCSI for internal storage, and SAS HBA external host interfaces are just starting to ship in volume. SAS disk arrays started shipping this year and SAS JBOD are starting to ship. To complement the SAS HBA's and SAS arrays, there are new network components called expanders which are basically switches. Generally SAS disk will be cheaper than Fibre Channel (FC) with similar performance and capacity. (Sun has SAS host I/F, disk I/F and SAS disk drives)
. iSCSI: In the SAN market, the iSCSI protocol is making a significant impact by taking the cost out of expensive Fibre Channel SANs. iSCSI allows users to send SCSI commands over their existing IP networks.
. File Virtualization: In the NAS market, file virtualization is making NAS farms much more scalable and manageable (See trend #2)
. Clustered RAID: A new innovation to watch in the RAID market is horizontally scalable RAID or clustered RAID for large applications. Digi-Data is a small storage company pushing this innovation.
. JBOD: Keep your eye on Sun in the JBOD market. JBOD is more affordable than RAID, but does not have RAID’s redundancy and reliability features. But what if you had an infinitely scalable file system with data integrity and RAID features running on JBOD, say, something like ZFS?
. Unified Storage: Another trend in this area are unified or hybrid devices – storage systems that can handle multiple protocols and interfaces, including iSCSI, Fibre Channel, and NAS – all in one unit. Makes sense to users with a dizzying array of choices in the market.
. 10GbE: Lastly, 10 Gigabit Ethernet or 10GbE is the latest and fastest of the Ethernet standards that will re-shape data center networking – offering a fast, common and affordable network technology for IT and Web 2.0 applications from supercomputing to networked storage.
9. Solid State Disk (aka SSD, Flash, Memory): We see the perfect storm happening around SSD. SSD has no moving parts, making it one of the most reliable and fastest storage mediums in the world. But it is incredibly expensive when you compare $/GB vs. Disk and Tape storage. However, SSD has made a name for itself in the consumer market (digital cameras, phones, iPods, etc.) The price pressure in the consumer arena is enormous – and this had been rapidly eroding the price of SSD. As SSD prices drop, expect to see hybrid disk drives and storage systems that leverage more SSD for greater speed and IOPS.
10. Storage as a Service:
Storage as a service offered over the Internet has been talked about for years – but poor performance and implementations have cooled this trend. However, Amazon has given Storage as a Service a power boost with its Amazon Simple Storage Service (Amazon S3). By leveraging Amazon’s existing e-commerce and storage infrastructure, the company is offering customers storage capacity for $0.15 per GB-Month of storage used – possibly the cheapest $/GB on the planet. And while this may have more play in the consumer market, Amazon could re-invigorate the storage as a service trend. Also keep an eye on Sun’s Internet Service offerings over Network.com … Continue Reading »
Subscribe to:
Posts (Atom)