The second day in September marked an opportunity for Amazon Web Services Inc. to hold its third annual Storage Day event, which included exclusive insights and announcements from leaders across AWS Storage.
TheCUBE’s coverage of AWS Storage Day featured interviews with key executives from the cloud provider and other firms, who shared a rich perspective on the state of the storage industry. Here are five insights you might have missed from the hours of interviews by theCUBE. (* Disclosure below.)
1. The AWS granular approach for its products and services pays big dividends.
Deep into theCUBE’s interview with Wayne Duso, vice president of engineering at AWS, can be found an important remark about the company’s strategic approach, which speaks volumes about how the cloud provider has become so immensely successful.
Duso, who was interviewed as part of the firm’s announcement to run NetApp Inc.’s full ONTAP stack natively in the cloud as a managed service, described the philosophy of former AWS chief and newly elevated Amazon CEO Andy Jassy.
“I remember Andy Jassy saying: ‘Hey, we purposefully have created our services at a really atomic level so that we can get down to the primitives and changes as the markets change,’” Duso recalled. “To me, that’s an opportunity for builders to create abstraction layers on top of that. In general, your philosophy is to maintain that agility at the really granular level.”
AWS recognized early in its history that builders need to build. It was very important that the products it released reflected this philosophy and gave developers an opportunity to create. It’s also why Amazon has transformed storage. The rigid nature of storage appliances doesn’t apply in the cloud, where file systems can be spun up in seconds and maximum agility defines the value proposition.
2. From finance to fast food to football, modern business is now cloud-driven at mammoth scale.
On the surface, there may not seem to be a lot of common ground between oversight of financial trading, ordering a hamburger, or identifying a collision between two National Football League players on a field. In fact, all of these share a common bond in being driven by the cloud at scale.
Much of the cloud’s impact has become transparent. Regular customers at a number of McDonald’s drive-thru windows now receive a customized menu to help decide what to order, based on a careful analysis of their previous choices. McDonald’s feeds 68 million people per day.
The NFL is working with AWS to develop a Digital Athlete in an effort to reduce player injuries. A computer vision competition last year yielded 7,800 submissions from data scientists for new tools in detecting on-field helmet impacts.
Yet, the most astounding example of how the cloud has transformed tasks among business sectors at scale may well involve the use case of the Financial Industry Regulatory Authority. Authorized by Congress to protect investors and ensure stock market integrity, FINRA moved its key monitoring technology to the cloud to perform 500 billion validation checks daily, according to AWS.
“They’re choosing to go to the cloud on the most important thing that their business does because they must; they have to,” said Mai-Lan Tomsen Bukovec, vice president of AWS Storage, in an interview with theCUBE. “FINRA was looking at the rate of stock transactions going up and saying: ‘We just can’t keep up, not if we want to follow the mission of being the watchdog for consumers, for stock transactions.’ And so they moved that crown jewel of their application to AWS.”
3. Serverless has become a key part of the AWS storage strategy.
It sounds like the ultimate paradox. How can serverless play a central role in the way Amazon executes its master plan for storage?
The answer can be seen in two of theCUBE’s interviews as part of Storage Day. Kevin Miller, vice president and general manager of S3 at AWS, singled out S3 Object Lambda, which was quietly launched in March, as an example of where the firm is seeking to streamline the use of serverless technology.
The new solution allows developers to add code while retrieving data from S3 before it goes back into an app, supplanting previous methods that added complexity and cost.
“It really is cool technology,” Miller told theCUBE. “We’re starting to see how it can be applied to simplify serverless application development. Not only is the storage fully managed, but the compute is fully managed as well.”
The other hint of AWS’ focus on serverless can be found in theCUBE’s conversation with Duncan Lennox, general manager of Amazon Elastic File System. The EFS offering does not require compute provisioning to use and is designed to provide absolute simplicity for the user, according to Lennox.
“It’s our simple serverless ‘set and forget’ elastic file system service,” Lennox explained. “The serverless part of it is there’s absolutely no infrastructure for customers to manage; we handle that entirely for them. You can also mange your file system from any of our compute services in AWS, not only directly on an EC2 instance, but also from our serverless compute models like AWS Lambda and Fargate.”
4. Amazon is part of a village currently nurturing the world’s most popular file system for high performance computing.
In describing FSx, Amazon’s highly performant file system, Ed Naim, general manager of Amazon FSx at AWS, made sure to mention that the firm offered a version specifically tailored for Lustre. Naim’s reference was for a good reason. Lustre has become the most popular high-performance file system, used on 7 out of the top 10 fastest computers in the world today and on 70% of the top 100 systems.
Despite its current popularity, Lustre has ventured down a rocky road for survival. The file system, named for an open-source blending of Linux and cluster, offers a parallel distributed architecture that can support multiple clusters and thousands of nodes. It was born out of an academic project and sold to Sun Microsystems, which in turn was acquired by Oracle Corp.
Lustre then was broken up and distributed among several companies, including Intel, which sold the technology off to DataDirect Networks Inc. in 2018, where it remains today.
When clusters grew in importance in the enterprise, the reliance on Lustre grew with it. The file system is used in a wide range of critical HPC environments today, including oil and gas, seismic processing, and scientific research. It will be interesting to see how AWS continues to develop its storage offerings, like FSx, to support the Lustre community, given the dependence of major industries on the file system technology.
5. In the infinite archive, most of the world’s data is cold as ice.
In an interview on theCUBE last year with Fred Moore, president of consulting firm Horison Information Strategies and author of the paper “Reinventing Archival Storage: the Storage Hierarchy,” the storage guru outlined his vision of the archival future. In the growing repository of the world’s data, the largest storage class will be cold, seldom used, and unlikely to ever be deleted. This will correspond to 60% of all data, reaching as high as 80% in three years, according to Moore.
“It’s infinite storage,” Moore said. “I mean, if you look at major league sports, the World Series and Super Bowl, that data will never be deleted; it’ll be here forever. It’ll be used periodically based on circumstances.”
Major cloud players such as AWS are aware of this reality and are positioning for it. In March, AWS made significant cuts in its pricing for S3 Glacier, the cloud provider’s archival storage service. A month later, the company announced the Glacier Re:Freeze serverless service, allowing customers to transfer a Glacier data vault to a different class of S3 like Glacier Deep Archive.
The key takeaway here is that an important part of AWS’ long-term storage strategy involves positioning the cloud for providing archival support as cold data continues to grow, and that could become a major cost saving opportunity in the enterprise.
“Most of the world’s data is cold by nature; it’s not data that you’re accessing all the time,” said Joe Fitzgerald, general manager of Amazon S3 Glacier at AWS, in an interview with theCUBE. “Every industry definitely has archived data, and we have customers in every vertical segment. If you don’t have an archival story as part of your data strategy, I think you’re missing out on a cost savings opportunity.”
(* Disclosure: TheCUBE is a paid media partner for the AWS Storage Day. Neither Amazon Web Services Inc., the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)