Compute
The processing capability of a system, used to execute instructions, run applications and transform data into results.
-Compute processes a website request
-Calculates a spreadsheet formula
-Runs an AI model.
In the cloud, you access compute through models such as virtual machines, containers and serverless services, running on processors such as CPUs and accelerators like GPUs.
Basics of Compute concepts, with a focus on cloud compute services.
Compute transforms inputs into outputs by executing instructions. Performance depends on the whole system, not just the advertised speed of a processor.
The CPU handles general instructions and control flow. Memory (RAM) holds active data. Storage retains programs and datasets. Networking moves data between systems. An accelerator executes selected work efficiently when software can use it. Running out of any one resource can limit the application.
A vCPU is a virtual CPU allocation, not a universal unit of physical performance. On many instance types it corresponds to a hardware thread; on others the relationship differs. Clock speed, processor generation, instruction set and memory bandwidth also matter.
More CPUs are NOT ALWAYS helpful
Suppose a request spends 20 ms computing and 180 ms waiting for a database. Making computation twice as fast reduces total latency from 200 ms to 190 ms, a 5% reduction. This illustrative calculation points toward database or network investigation before buying more CPU capacity. Optimization should remove the dominant bottleneck while preserving correctness and the required user experience.
On the cloud, you usually access virtual resources, not physical hardware. Physical resources are shared between multiple customers (tenants).
VMs are an important pillar of compute services on the cloud.
A virtual machine presents an isolated, machine-like environment to an operating system. Multiple VMs can use one physical server while retaining separate guest operating systems.
A hypervisor manages virtual hardware and isolation. The guest OS schedules application processes as if it owned a machine; the virtualization layer maps its CPU, memory and device activity to physical resources. Hardware virtualization support reduces the need to emulate every instruction.
VMs support legacy software, custom agents, familiar administration tools and applications that need OS-level control. They provide a practical migration target when rewriting an application is more expensive than moving its existing environment. They also create boundaries for patching, resource allocation and failure investigation.
Amazon EC2 offers virtual instances and selected bare-metal instance types. Many EC2 families use the AWS Nitro System, which combines hardware offload with virtualization and security components. Bare metal exposes the physical server without a guest-VM hypervisor layer.
You normally administer the guest OS on an EC2 VM: install patches, secure accounts, configure services and manage application dependencies. AWS operates the underlying physical infrastructure. A VM boundary does not remove the need for least-privilege access or backups.
When building an application on AWS, EC2 can be used to supply compute capacity and used in combination with other services. A reliable application combines it with services for networking, storage, identity, traffic routing and operations.
Serverless Compute
Serverless means the provider manages more of the infrastructure needed to execute a
workload. Physical servers still exist, and application design still matters.
Instead of managing a fleet of guest operating systems, you deploy code or a workload definition and configure supported resources, permissions and triggers. Scaling and infrastructure maintenance shift toward the service. Your team continues to own code, dependencies, data handling, access rules and business correctness.
Function as a service is one serverless form. AWS Lambda invokes a handler for incoming
work. Serverless containers, such as supported Fargate workloads, keep a container-oriented
model while removing host management. Serverless also describes managed data and
integration services; it is not limited to functions.
Serverless does not mean zero configuration, unlimited scaling or automatic cost savings. There are quotas, scaling behavior and service-specific billing rules. Some services maintain paid capacity even when no user request is actively executing.
A long-running program, specialized device requirement or strict latency target may fit a
different execution service better. That is a workload decision, not evidence that one model is
universally more modern.
AWS Lambda
AWS Lambda is a serverless compute service that runs your code without requiring you to provision or manage servers. You package your application logic into a function and configure its memory, timeout, permissions, and triggers. AWS manages the execution infrastructure and automatically scales function execution as demand changes, within service quotas and your configured limits. Common uses include website backends, processing uploaded files, and automating scheduled tasks. Standard on-demand function pricing primarily depends on the number of requests and execution duration, with compute charges reflecting the configured memory.
A Lambda function starts when it is invoked, for example by an API request or a configured S3 upload notification. Lambda prepares or reuses an execution environment and passes event data to your function’s handler—the code that processes the request. The function uses its IAM execution role to access permitted AWS resources, such as S3 or DynamoDB, and can send logs to CloudWatch. For a synchronous invocation, the caller waits for the result; for an asynchronous invocation, Lambda queues the event for processing and the caller does not wait for the function’s result.
GPUs, Accelerators and AI Data centers
A Graphics Processing Unit (GPU) is a processor designed to perform many calculations in parallel. GPUs were originally developed to render graphics, where similar operations must be applied to large numbers of pixels and vertices. This design also makes them useful for scientific computing and artificial intelligence. A CPU generally excels at complex control logic and sequential tasks, while a GPU delivers high throughput when work can be divided into many similar calculations. Modern CPUs also support parallel processing, but their architecture and strengths differ from those of GPUs.
GPUs are particularly useful for deep learning, because neural networks perform large numbers of mathematical operations on arrays of numbers called tensors. Many of these operations involve matrix multiplication, which GPUs can accelerate efficiently. During training, a model processes examples, calculates a loss that measures its errors, and uses backpropagation to calculate gradients. An optimizer then uses those gradients to update the model’s parameters. GPUs accelerate these repeated calculations, helping make large neural networks practical to train. However, the speed improvement depends on the model, workload size, software, and hardware configuration.
After training, GPUs can also accelerate inference, which means using a trained model to produce predictions or generate content. Examples include recognising objects in images, transcribing speech, and generating a chatbot’s response. Inference normally uses the learned parameters without updating them. GPU acceleration can reduce response time and increase the number of requests a system handles, especially when requests are processed together in batches. Some GPUs include specialised hardware, such as NVIDIA Tensor Cores, for accelerating matrix calculations. Using lower-precision numerical formats can improve speed and reduce memory requirements, but developers must check that model accuracy remains acceptable.
A GPU’s usefulness for AI depends on more than its processing power. Memory capacity determines how much model data and working information it can hold, while memory bandwidth determines how quickly that information reaches the processing units. Large models may require multiple GPUs, making communication between devices another important consideration. Software frameworks and optimised libraries help applications use this hardware effectively. GPUs also work alongside CPUs, which coordinate execution and handle other application tasks. Consequently, a GPU is not automatically the best choice for every AI workload: small or mostly sequential tasks may gain little benefit, particularly when data-transfer overhead outweighs the accelerated computation.
Many machine-learning workloads repeat similar mathematical operations across large arrays. Accelerators can execute those operations efficiently when enough compatible work is available.
Training adjusts model parameters using examples and an optimization procedure. A
neural-network training step commonly performs a forward pass, computes a loss, calculates
gradients and updates weights. Inference applies learned parameters to new inputs. It usually
does not compute training gradients, but large models can still demand substantial memory and compute.
A tensor is a multidimensional array. A matrix is a two-dimensional tensor. Dense
neural-network layers often involve matrix multiplication plus other operations. For matrices of
sizes M x K and K x N, a conventional dense multiply requires approximately 2 x M x N x K
floating-point operations when multiplication and addition are counted separately.
Different output elements can often be computed concurrently. GPUs and AI-specific chips
provide many arithmetic units and data paths designed to exploit this structure. CPUs remain
important for input preparation, control flow, networking and work that does not map efficiently to the accelerator.
A tiny model can spend more time transferring data and launching kernels than calculating. An
application with slow preprocessing may leave the device idle. An unsupported operation may
fall back to another processor and introduce extra transfers. Some classical ML workloads and
small inference models can run efficiently on CPUs
An NPU (neural processing unit) is specialized hardware for neural-network computation. The term describes a broad category rather than one universal architecture or programming
interface.
Many phones, PCs and embedded systems include NPUs optimized for supported inference
workloads with constrained energy and thermal budgets. Examples include audio enhancement, camera effects, local vision and some local language-model features. Keeping eligible computation on the device can reduce dependence on network connectivity. This does not mean every NPU runs every model or trains large models efficiently. Device memory, supported operators, tensor shapes, precision formats, compiler behavior and driver support all affect the usable workload.
A framework or runtime converts a model into operations the device can execute. Some graphs
require transformation, quantization or compilation. Unsupported operations may cause failure or fallback to another processor, depending on the runtime. Check the actual execution placement instead of assuming that selecting “NPU” accelerates the complete application.
Intel OpenVINO is one example of a software stack with an NPU device backend.
