When thinking about edge orchestration and optimising all components, you must first define what you mean by “edge”, according to Nick Burling, chief product officer (CPO) at hybrid cloud storage-focused company Nasuni. Your “edge” can be a cloud region, or a central location where data must be accessible. Or is it your “edge” because compute is only available in that region?
For Nasuni customers, “edge” can be a military installation or a North Sea exploration vessel scanning subsurface geology. The datasets requiring efficient, economical storage can be enormous LiDAR (light detection and ranging) scans of the seabed, says Burling.
There must be a “gold” master copy of the data somewhere, whether you’re talking about the 80% of enterprise data that’s unstructured, or another data type. Data may need to be presented in different subsets or a different way per different edge locations, and be available for analysis, which means sufficient connectivity, availability, latency and tolerance of disruption.
Containerisation can be used to provide modularity and redundancy for flexibility and future-proofing.
“Caching is absolutely critical,” says Burling. “Looking through a hardware-centric lens, we separate compute orchestration from data orchestration to avoid big silos of copies of data.”
For example, the system may need to present a specific required subset of data to an analyst to be worked on elsewhere without having to duplicate that data. And data orchestration done right can help avoid expensive physical storage or having to continually transmit data back and forth.
In addition, object storage and peer-to-peer networking can work together to deliver centralised data management and protection while presenting wherever the edge might be, with data orchestration layered on top.
“Ideally, independent of whatever local storage you have at those locations, as long as you can present the smallest amount of the working set of the data that’s needed, that’s where you get to an optimised workflow,” says Burling.
Small is not beautiful
An edge device, such as a sub-100W device, uses a third of the power of a standard PCI Express graphics processing unit (GPU). Such a device needs to balance reasonable performance with thermal limitations, as Ed Plowman, CTO at Imagination Technologies, explains.
“Traditional thinking, particularly around hardware compute for AI processing, is that you have a CPU [central processing unit], a GPU that draws pretty pictures and does some of the compute work, and then an NPU [neural processing unit], because NPUs are low power,” he says. “But it means programming three elements and getting them to cooperate all at once.”
When looking at edge compute, it is necessary to take into account performance, throughput and capability for devices and CPU, GPU and NPU software stacks, as well as the maturity of the software.
The experts Computer Weekly has spoken to urge IT decision-makers to compare their requirements to designs that work, and make use of programmable, easy-access, easily understood open source capabilities to avoid being locked into a technical architecture.
“Stay flexible, so you can hopefully continue to develop what you do and choices continue to be fit for purpose,” Plowman urges.
Joshua David, senior director of product management – edge business expansion at Red Hat, recommends that IT decision-makers frame their edge compute strategy around orchestration and how to manage multiple distributed, resource-limited sites without adding on extra cost, complexity or risk.
Edge workloads run differently from datacentre workloads and require real-time control automation, or artificial intelligence (AI) inferencing that needs to run close to or on the edge device itself. Rather than focusing on moving compute closer to data, IT decision-makers should ensure their edge compute IT architecture is optimised in terms of infrastructure type, workload size, latency, bandwidth, reliability and security.
Coordinate and centralise
Site-level infrastructure that can coordinate these local operations typically provides centralised management with enterprise analytics or AI training and fleet-wide visibility.
“For example, manufacturing may prioritise low-latency machine control. Utilities may require autonomous operations. Retailers may focus on store load experiences with centralised analytics,” David advises. “These are distinct. So start with the type of workloads, then those workloads’ characteristics, and then specific industry needs.”
Another consideration is the choice of compatible enterprise operating system (OS) and bare-metal hardware that runs at the edge. For instance, edge-optimised, open source-derived approaches (such as MicroShift) can add flexibility and interoperability.
Workloads may run as a virtual machine, a container, or need hardware acceleration for real-time capabilities. IT decision-makers should also ensure that observability – in terms of notifications and alerting – is robust and able to respond to relevant shift or drift, ideally via a central control plane observability tool.
This leads on to reliability and ensuring business continuity at the edge. Mark Christie, field CTO at StorMagic, says customers such as retailers may generally require a two-node high-availability configuration. “What you’re seeking is an elasticity that smoothly transitions from ‘one to two to many’ as required,” he says.
For example, US convenience store chain Sheetz uses a two-node system with 15 applications, from Windows operating systems to LTSC for credit, loyalty and point-of-sale applications. The storage “stretch” cluster can be shared out to as many compute nodes as it needs.
Beyond observability, an edge control cloud provides centralised management for edge orchestration of fleets, estates and multiple edge device clusters. It tends to run as a serverless tenancy in the cloud. The orchestrator transmits information about the infrastructure to the cloud, which provides centralised management, and acts as a single source of the truth for edge compute IT infrastructure. Having this self-hosted helps tick data sovereignty and residency boxes for regulatory compliance.
AI at the edge
Adding intelligence is complex since it is necessary to ensure edge AI workloads run reliably across distributed devices with minimal data movement.
Andreas Hellander, CEO and co-founder at edge AI-focused company Scaleout Systems, considers edge compute requirements through the lens of federated machine learning (ML).
“There are so many aspects of or challenges related to deploying, updating and continuously improving AI systems on the edge,” he says. “And it’s a little bit fluid. It’s not so easy to be completely strict about where one layer in this hierarchy starts, and where the other one stops.”
Beyond the IT infrastructure hosted in a hyperscale cloud, IT decision-makers need to look at where near-edge resources are hosted. They may be hosted in private but not centralised cloud infrastructure. Then there is the far edge, with edge workstations, gateway nodes and devices like personal computers, smartphones and drones.
What all of this means is more environments need supporting and the further out to the edge AI workloads need to run. “Everything is different: hardware, OSes, connectivity,” says Hellander. While the ecosystem to support heterogeneity at the edge has matured in the past five years, Hellander warns that a setup that works when it is first deployed may use integration that later becomes obsolete or cannot easily be updated.
“When dealing with security-sensitive or defensive cases, you want to control the ML lifecycle. If you don’t, you’re really taking on security risks and technical debt,” he says. “And it’s always important to ask, ‘Where does my data sit and end up, and what flexibility do I have to land it on a far or near edge node or central cloud?’ Also, ‘What about resilience? What happens if the network is cut?’”
What seems to be apparent from Computer Weekly’s conversations with industry experts is that edge computing complicates centralised IT management and presents observability and business continuity problems. AI complicates this further, especially as edge devices need to balance performance, power consumption, access to local data and potentially consolidated datasets with having enough compute to run inference workloads.
S 004