NVIDIA H100
H100 SXM5 is the Hopper datacenter GPU: 80 GB HBM3, 3.35 TB/s, fourth-generation Tensor Cores, and FP8 through the Transformer Engine. A typical SXM node quote is 26 vCPUs, 116 GB of host RAM, and 80 GB of VRAM. It is still the default card for 30B–70B training and production FP8 serving.
Implante NVIDIA H100 do seu jeito: pods Docker ou VPS KVM root
A NovaPod deixa você escolher como usar a GPU. Um contêiner já em cache, uma máquina virtual root com IP público ou um cluster NVLink de vários nós: você paga só os segundos em que roda, em cripto.
Contêineres de IA leves e rápidos
Computação de GPU sem o peso do hipervisor. Entre direto no PyTorch, vLLM, ComfyUI ou JupyterLab.
- Medição com precisão de 1 segundo
- Mais de 37 modelos de framework de ML já em cache
- Sem taxa ociosa quando a instância está parada
Máquina virtual root completa
Sistema dedicado (Ubuntu 24.04 LTS / Windows GPU Server) com SSH root e sudo desde o primeiro minuto.
- IPv4 público dedicado (todas as portas abertas)
- Pause quando quiser; pague só US$ 0,00014/GB/h pelo disco
- Rode Docker-in-Docker, k3s e drivers próprios
Supercomputação e treino distribuído
Junte até 8× NVIDIA H100 em um nó com InfiniBand Quantum-2 RDMA para escalar a vários nós.
- Troca de gradiente all-reduce sem perda
- Pronto para PyTorch FSDP e DeepSpeed Zero-3
- GPUDirect Storage nativo e malha RDMA
Especificações de hardware
- Architecture
- NVIDIA Hopper
- VRAM
- 80 GB HBM3
- Bandwidth
- 3.35 TB/s
- Tensor Cores
- 528, 4th generation
- CUDA cores
- 16,896
- FP64
- 34 TFLOPS
- FP32
- 67 TFLOPS
- TF32
- 989 TFLOPS with sparsity
- FP16 Tensor
- 1,979 TFLOPS with sparsity
- FP8 Tensor
- 3,958 TFLOPS with sparsity
- INT8
- 3,958 TOPS with sparsity
- NVLink
- 900 GB/s
- MIG
- Up to 7 instances
- Reference host
- 26 vCPUs, 116 GB RAM
- Form factor
- SXM5 or PCIe
- TDP
- 700 W SXM5, about 350 W PCIe
- Multi-node
- 400 Gb/s class fabric on serious jobs
Still the default datacenter GPU
H100 SXM5 is 80 GB of HBM3 at 3.35 TB/s, 16,896 CUDA cores, fourth-generation Tensor Cores, and FP8 through the Transformer Engine. FP32 is 67 TFLOPS. FP64 is 34 TFLOPS, about three times an A100, which is why some simulation codes still ask for Hopper specifically. Board power on SXM5 is 700 W. The PCIe card is about half that and has no NVLink.
The catalog host next to this GPU is 26 vCPUs and 116 GB of RAM. That is one training process and its dataloader, not an eight-GPU chassis. Multi-GPU training wants SXM boards on NVLink inside the node, and a 400 Gb/s class fabric with GPUDirect RDMA once you leave the node.
What fits in 80 GB
A 70B model in FP8 fits on one H100 with on the order of 10 GB left for KV cache. That is enough for moderate batch and moderate context. A 30B model fits in BF16. FP16 70B does not fit. Long context or a large batch pushes you to H200 at 141 GB.
Decode at batch size one is bandwidth-bound. H100's 3.35 TB/s is about 1.6× an A100 80GB, which is most of the inference gap people feel before they look at FLOPS. FP8 on the Transformer Engine is roughly a 1.6× step over FP16 on attention and MLP layers when the software path is actually on.
How a job is launched here
Open the marketplace, take an H100 offer, and start from a vLLM, PyTorch, or CUDA template. Billing is the hourly ask divided into seconds. There is no 20-minute minimum and no separate spot product unless the offer itself says the host can reclaim the card.
A QLoRA fine-tune of a 70B model on a modest dataset is often a working day on one H100. A full-parameter fine-tune wants several GPUs and checkpoints. Frameworks you should expect to work are PyTorch, TensorFlow, JAX, vLLM, TensorRT-LLM, and SGLang, on a CUDA 12 image.
SXM, PCIe, and clusters
Choose SXM when GPUs in the node must talk over NVLink. Choose PCIe when you are serving one model and the host is a normal server. Mixing them in one training job is how you get a slow rank.
Past one node, all-reduce time dominates unless the network is built for it. Plan InfiniBand or a comparable RDMA fabric, NCCL that matches the topology, and a checkpoint interval you can afford to lose. Eight GPUs on NVLink is a different design from eighty GPUs on ethernet.
O que os números significam em produção
- Bandwidth vs A100 80GB
- ~1.6×
- Training vs A100
- About 3–4×
- 70B FP8
- One GPU
- FP64
- 34 TFLOPS
3.35 TB/s versus about 2.0 TB/s
On FP8 transformer workloads, not on every kernel
Weights plus a modest KV cache
About 3× A100, relevant for simulation
Comparação entre gerações
Pick on the constraint. H100 is the compute default. H200 is the memory upgrade. A100 is the cheaper Ampere card.
| H100 | H200 | A100 80GB | |
|---|---|---|---|
| Architecture | Hopper | Hopper | Ampere |
| VRAM | 80 GB HBM3 | 141 GB HBM3e | 80 GB HBM2e |
| Bandwidth | 3.35 TB/s | 4.8 TB/s | 2.0 TB/s |
| FP8 | Yes | Yes | No |
| Menu host | 26 vCPU · 116 GB | 44 vCPU · 182 GB | 14 vCPU · 100 GB |
Armazenamento rápido, portas nativas e trilhos cripto
Cada instância NVIDIA H100 na NovaPod se conecta a uma infraestrutura de alto débito feita para tirar gargalos de E/S e a espera inicial.
Armazenamento de rede compartilhado a 25 Gb/s
Monte checkpoints e conjuntos de treino em uma ou várias instâncias ao mesmo tempo. Os dados duram além de um pod e continuam acessíveis quando as instâncias param.
JupyterLab, VS Code & SSH
Conecte do terminal local com chaves SSH comuns ou abra IDEs na web em abas do navegador, sem configurar proxy reverso.
Trilhos multi-chain sem custódia
Deposite USDT, USDC, SOL, ETH ou BNB no seu endereço pessoal. A confirmação on-chain leva segundos e não há verificação de identidade obrigatória.
Cargas indicadas para NVIDIA H100
Training and fine-tuning
FP8 mixed precision is why H100 still beats A100 on 30B and larger runs. A modest 70B QLoRA on one card is often a working day.
Production inference
A 70B model in FP8 fits on one SXM with some KV-cache room. FP16 70B does not fit.
Diffusion and HPC
Batch image and video jobs, plus FP64 work that is about three times an A100.
Multi-GPU training
Stay inside one SXM node on NVLink until the batch no longer fits. Past that, all-reduce wants a real fabric.
Quando escolher NVIDIA H100
01
Pick H100 for FP8 and 30B and up
The Transformer Engine is the reason to pay more than an A100.
02
Pick A100 if you do not need FP8
Sub-30B training, BERT-scale jobs, and many inference paths are cheaper on Ampere.
03
Pick H200 if you run out of memory
Same compute family, 141 GB of HBM3e when the KV cache or a colocated model does not fit.
04
Pick PCIe only for one GPU
SXM5 is 700 W with NVLink. PCIe is lower power and has no NVLink. Do not mix them in one training job.
SXM5 versus PCIe
SXM5 is 700 W with NVLink inside the node. PCIe is about 350 W and easier to mix into an existing server. Train on SXM. A single inference worker can be PCIe.
Multi-node
Serious all-reduce wants a 400 Gb/s class fabric and GPUDirect RDMA, not the datacenter ethernet you use for SSH. Stay on one node until the batch no longer fits.
Perguntas frequentes
How much does an H100 cost here?
What is the cheapest path?
Can I rent it by the hour?
How fast does it start?
SXM or PCIe?
Does it do FP8?
Will 70B fit?
Do you support clusters?
Which regions?
Which frameworks?
Is it worth it over A100?
Rent or buy?
Outras opções de hardware
Pronto para implantar NVIDIA H100?
Inicie uma instância no mercado ao vivo em menos de 15 segundos. Abasteça o saldo com criptomoeda e pague por segundo, sem compromisso.
Iniciar no mercado →