Large-scale AI training depends on a communication fabric designed to move data with very low latency and high throughput. Technologies such as RDMA, InfiniBand, RoCE, GPUDirect, and collective-communication libraries can move data across boundaries that conventional application-security controls do not fully describe.
This research asks a practical question: what security assumptions change when performance-sensitive infrastructure can move data directly between accelerators, host memory, network interfaces, and storage?
Why the interconnect matters
Distributed training is not one system. It is a set of tightly coupled components:
- accelerators and their local memory;
- host operating systems and drivers;
- network interface cards and switching fabric;
- collective-communication libraries such as NCCL and RCCL;
- schedulers, orchestration, storage, and monitoring systems.
The security boundary is defined by how these components establish identity, authorize memory access, isolate tenants and jobs, and expose activity to defenders.
Threat-model scope
The current work is organized around four areas.
Memory and data movement
RDMA-capable systems are built to avoid copies and operating-system intervention. The research examines where those performance properties create opportunities for unintended memory exposure, unauthorized transfers, or weak assumptions about which component is allowed to access a buffer.
Isolation between jobs and tenants
Training clusters may host workloads with different owners, sensitivity levels, and trust assumptions. Relevant questions include how isolation is enforced across queue pairs, protection domains, device state, collective-communication groups, and the lifecycle of a scheduled job.
Control-plane trust
The data path depends on configuration and orchestration. Compromise or misuse of schedulers, drivers, management interfaces, topology services, and communication libraries may change what the high-speed fabric is allowed to do without attacking the fabric directly.
Security observability
Traditional network monitoring relies on visibility that high-throughput or direct-memory paths may not provide. The research looks for evidence that can support detection and investigation without imposing unacceptable performance costs.
Questions guiding the work
- Which components can authorize or initiate memory movement, and how is that authority scoped?
- What state persists between jobs, devices, or tenants?
- Where can identity or topology information be spoofed or confused?
- Which failures can cross from one accelerator or host into a wider training job?
- What telemetry is available at the device, host, fabric, and orchestration layers?
- Which controls can prevent, contain, or detect misuse while preserving training performance?
Current direction
The goal is to produce a threat model that connects systems-level behavior to practical security controls: stronger isolation, narrower authority, safer lifecycle management, and useful detection points.
This research is a collaboration with Matt Schultz and Ilil Blum Shem-Tov.