AirLLM Runs 70B Inference on a Single 4GB GPU

The AirLLM project demonstrates running inference on a 70-billion-parameter model using a single 4GB GPU, by streaming layers through the limited memory available rather than requiring the whole model resident at once. It lowers the hardware floor for anyone experimenting with large open models.