As much as we try, it’s impossible to defeat physics. I’m speaking specifically of data gravity and compute. The internet of things on the edge of the network has a dramatic impact on IT operations and vendor services.
For example, internet of things (IoT) forced a change in data patterns, prompting Amazon to release Snowball Edge — a data transfer device with integrated storage and compute resources. Snowball Edge enables some computing capability on the edge of the network and facilitates the transfer of data back to the cloud. Is Snowball Edge a representation of the future of public cloud? Andreessen Horowitz partner Peter Levine argues that IoT will bring an end to public cloud.
Data gravity
Levine’s argument is that the growth in the number of sensors and the need to process that data will choke rather than accelerate the adoption of public cloud. The public cloud model assumes data resides within the cloud provider’s data center. A consistent complaint is that data gravity plays in favor of Amazon Web Services (AWS) rather than the customer.
Amazon provides a multitude of methods to load data into AWS. From physical Snowball drives to storage on tractor-trailers, AWS ingests data of any size. The cost to get data into AWS is minimal compared to the cost to get data back out. It’s expensive to leave AWS, a reality that makes data heavy.
There are ample advantages to hosting data within the AWS data center. By far, the most appealing feature is the ability to leverage the ever-growing list of AWS compute services against all the data you’re collecting. AWS now provides machine learning or artificial intelligence as a service. For practical use, machine learning requires a massive amount of data. It’s this data set and the processing that Levine views as the death of cloud computing.
Many mini data centers
IoT-enabled devices continue to generate a massive amount of data, and data gravity plays a critical role in the ability to use data generated by sensors. An example is the data needed for self-driving cars. The sensors on these cars ingest terabytes of data. Self-driving cars must process sensor data in real time. From a pure latency and bandwidth perspective, it’s impossible to transmit data from millions of cars back to a data center for decision processing.
Therefore, the decision processing must occur closest to the data, within the car. It takes significant processing power to identify a stop sign in inclement weather conditions and decide the best course of action based on road conditions. This example isn’t unique to self-driving cars. IoT edge computing is starting to scale at a similar rate to the data collector at the edge. According to Levine, today’s non-self-driving luxury car has 100 CPUs. Public cloud simply can’t counteract the effect of physics for real-time IoT processing.
What does this all mean for enterprise IT and public cloud adoption? IoT is the expected next boom in both consumer and enterprise computing. A single business may have millions of sensors generating and processing terabytes of data on local devices. Each local device is the management equivalent of a data center or at least a data center pod. As a result, IoT presents enterprise IT with unique challenges.
IoT edge computing infrastructure
AWS Snowball Edge is an admission that some level of computing occurs at remote sites. The example given at AWS re:Invent 2016 was wind farm sensor data. Turbines generate a tremendous amount of data, some of which requires analysis close to the source of generation. Snowball Edge is capable of performing some analysis and shipping the larger data set back to AWS for data mining or archiving. In many cases, the compute resources required to process IoT data resemble a data center’s worth of equipment.
The result? All of the problems public cloud attempts to solve reappear. Over the years, enterprise IT organizations focused on consolidating data centers and services. The consolidation of services created the opportunity for public cloud. It doesn’t matter to end users whether a consolidated data center is owned and operated by the customer or a public cloud provider. For a business, edge IoT computing reintroduces the complexities of distributed enterprise data center services.
In today’s environment, people, processes and technology are geared toward centralized management. Enterprise shops need to develop a strategy for deploying micro data centers. I predict that AWS isn’t going to let this business go to traditional enterprise IT vendors. Snowball Edge is a precursor to the types of hybrid platforms that address the capabilities of private clouds. I expect more managed physical hardware products that bring together Amazon’s vision of IoT, which includes its IoT software development kit.
I also expect vendors to double down on a trusted model of integrated platforms, such as hyper-converged infrastructure. I’m watching for IoT-specific investment. Cisco recently announced its purchase of AppDynamics, which will become part of Cisco’s IoT business unit. Prior to the Cisco purchase, AppDynamics hadn’t focused on IoT. The purchase and repositioning to the internet of things demonstrates that IoT is the next battlefront for enterprise IT vendors.
It’s too early to say that public cloud is at its end. I’ve abandoned the practice of beating against AWS. I do expect the offerings from both traditional companies and public cloud companies to adapt as enterprise IT adjusts to support IoT edge computing.
