NovaPodWorkload Playbook
LoRA fine-tuning on rented GPUs
Use this flow when you want cheap iteration speed first, then scale once the run is stable. The goal is to reduce failed long runs by validating data, tokenizer, and trainer settings early.
Before you launch
- - Dataset shard and tokenizer are prepared before launch.
- - Start with one GPU and short sequence length for smoke test.
- - Disk budget includes checkpoints, logs, and merged adapters.
- - Keep an API key ready if the run is managed by an agent script.
Execution steps
STEP 01
Start from a LoRA-friendly template
Pick an Unsloth or PyTorch template in Deploy so CUDA and trainer dependencies are preconfigured.
STEP 02
Run a 5-10 minute validation epoch
Verify loss decreases, checkpoints write correctly, and output format matches your downstream inference.
STEP 03
Tune memory and throughput
Adjust batch size, gradient accumulation, and sequence length to fit VRAM without OOM restarts.
STEP 04
Scale only after stable baseline
Move to larger VRAM or multiple GPUs only when a single-GPU run is reproducible end to end.
Pre-warmed Container Cache & 1s Crypto Settlement
Base CUDA runtimes and foundational weights are distributed across verified high-bandwidth nodes. Boot instances in under 15 seconds with zero KYC and pay with USDT, USDC, SOL, or ETH.
Budget and reliability tips
- - Compare ask versus median before long jobs; outliers can be cheap but unstable.
- - Use restartable checkpoints so spot-like interruptions are survivable.
- - Stop idle rentals immediately after adapter export to avoid overnight burn.