apextoaster
/
onnx-web
1
0
Fork 0
Code Actions

feat(api): log CUDA memory stats (#141)

This commit is contained in:
Sean Sube 2023-02-16 23:51:17 -06:00
parent 03a579a535
commit 005650a9a2
Signed by: ssube
GPG Key ID: 3EED7B957D362AF1
1 changed files with 6 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
api/onnx_web/utils.py
View File

@ -144,3 +144,9 @@ def run_gc(devices: List[DeviceParams] = []):
with torch.cuda.device(device.torch_str()):
torch.cuda.empty_cache()
torch.cuda.ipc_collect()
mem_free, mem_total = torch.cuda.mem_get_info(device=device.torch_str())
logger.debug(
"remaining CUDA VRAM usage: %s of %s",
(mem_total - mem_free),
mem_total,
)