Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 77 additions & 0 deletions .github/workflows/test-xpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Tests (Intel XPU)

# Opt-in workflow for the Intel XPU backend. Trigger it manually, or add the
# `xpu` label to a pull request. Repositories that do not use Intel GPUs are
# unaffected, and `test.yml` is untouched.

on:
workflow_dispatch:
pull_request:
types: [labeled, synchronize]

permissions:
contents: read

jobs:
# Runs on a GitHub-hosted runner with no Intel GPU attached. The XPU-enabled
# torch build ships from a separate index, so this checks that chgnet imports
# and selects a device correctly against it when `torch.xpu.is_available()`
# is False.
test-xpu-build:
if: >-
github.event_name == 'workflow_dispatch' ||
contains(github.event.pull_request.labels.*.name, 'xpu')
runs-on: ubuntu-latest

steps:
- name: Check out repo
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install chgnet against the XPU torch build
run: |
pip install torch --index-url https://download.pytorch.org/whl/xpu
pip install -e .[test]

- name: Check device selection with no Intel GPU present
run: |
python - <<'EOF'
import torch
from chgnet.utils import determine_device, gpu_devices_sorted_by_free_mem

print("torch", torch.__version__, "xpu", torch.xpu.is_available())
assert determine_device() == "cpu"
assert gpu_devices_sorted_by_free_mem("xpu") == []
EOF

- name: Run tests
run: pytest --capture=no
env:
CHGNET_DEVICE: cpu

# Requires a self-hosted runner labelled `intel-gpu` with real hardware.
test-xpu:
if: >-
github.event_name == 'workflow_dispatch' ||
contains(github.event.pull_request.labels.*.name, 'xpu')
runs-on: [self-hosted, intel-gpu]

steps:
- name: Check out repo
uses: actions/checkout@v4

- name: Report XPU visibility
run: |
python -c "import torch; print('torch', torch.__version__); print('xpu', torch.xpu.is_available(), torch.xpu.device_count())"

- name: Install chgnet
run: pip install -e .[test,logging]

- name: Run tests on XPU
run: pytest --capture=no
env:
CHGNET_DEVICE: xpu
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ if PyPI installation fails or you need the latest `main` branch commits, you can
pip install git+https://github.com/CederGroupHub/chgnet
```

CHGNet runs on CPU, NVIDIA GPUs (CUDA), Intel GPUs (XPU) and Apple silicon (MPS).
The device is picked automatically and can be overridden per call with
`use_device=...` or globally with the `CHGNET_DEVICE` environment variable.

## Tutorials and Docs

[![2023-11-02-sciML-webinar](https://github.com/CederGroupHub/chgnet/assets/30958850/49fe7d89-bf47-4ea0-aca6-f9014d2f41b8)](https://youtu.be/Lm148F_1Dn4)
Expand Down Expand Up @@ -130,8 +134,9 @@ md = MolecularDynamics(
md.run(50) # run a 0.1 ps MD simulation
```

The MD defaults to CUDA if available, to manually set device to cpu or mps:
`MolecularDynamics(use_device='cpu')`.
The MD defaults to the available accelerator (CUDA, then Intel XPU), to manually
set the device to cpu, cuda, xpu or mps: `MolecularDynamics(use_device='cpu')`.
The device can also be set globally with the `CHGNET_DEVICE` environment variable.

MD outputs are saved to the ASE trajectory file, to visualize the MD trajectory
and magnetic moments after the MD run:
Expand Down
18 changes: 11 additions & 7 deletions chgnet/model/dynamics.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,12 @@ def __init__(
the pretrained CHGNet is loaded.
Default = None
use_device (str, optional): The device to be used for predictions,
either "cpu", "cuda", or "mps". If not specified, the default device is
automatically selected based on the available options.
either "cpu", "cuda", "xpu", or "mps". If not specified, the
default device is automatically selected based on the
available options.
Default = None
check_cuda_mem (bool): Whether to use cuda with most available memory
check_cuda_mem (bool): Whether to use the accelerator with most
available memory
Default = False
stress_weight (float): the conversion factor to convert GPa to eV/A^3.
Default = 1/160.21
Expand Down Expand Up @@ -201,8 +203,9 @@ def __init__(
optimizer_class (Optimizer,str): choose optimizer from ASE.
Default = "FIRE"
use_device (str, optional): The device to be used for predictions,
either "cpu", "cuda", or "mps". If not specified, the default device is
automatically selected based on the available options.
either "cpu", "cuda", "xpu", or "mps". If not specified, the
default device is automatically selected based on the
available options.
Default = None
stress_weight (float): the conversion factor to convert GPa to eV/A^3.
Default = 1/160.21
Expand Down Expand Up @@ -800,8 +803,9 @@ def __init__(
optimizer_class (Optimizer,str): choose optimizer from ASE.
Default = "FIRE"
use_device (str, optional): The device to be used for predictions,
either "cpu", "cuda", or "mps". If not specified, the default device is
automatically selected based on the available options.
either "cpu", "cuda", "xpu", or "mps". If not specified, the
default device is automatically selected based on the
available options.
Default = None
stress_weight (float): the conversion factor to convert GPa to eV/A^3.
Default = 1/160.21
Expand Down
8 changes: 5 additions & 3 deletions chgnet/model/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -705,10 +705,12 @@ def load(
- "r2scan": R2SCAN level model transfer learned from MP-R2SCAN dataset
Default = "0.3.0".
use_device (str, optional): The device to be used for predictions,
either "cpu", "cuda", or "mps". If not specified, the default device is
automatically selected based on the available options.
either "cpu", "cuda", "xpu", or "mps". If not specified, the
default device is automatically selected based on the
available options.
Default = None
check_cuda_mem (bool): Whether to use cuda with most available memory
check_cuda_mem (bool): Whether to use the accelerator with most
available memory
Default = False
verbose (bool): whether to print model device information
Default = True
Expand Down
8 changes: 5 additions & 3 deletions chgnet/trainer/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,12 @@ def __init__(
data_seed (int): random seed for random
Default = None
use_device (str, optional): The device to be used for predictions,
either "cpu", "cuda", or "mps". If not specified, the default device is
automatically selected based on the available options.
either "cpu", "cuda", "xpu", or "mps". If not specified, the
default device is automatically selected based on the
available options.
Default = None
check_cuda_mem (bool): Whether to use cuda with most available memory
check_cuda_mem (bool): Whether to use the accelerator with most
available memory
Default = False
wandb_path (str | None): The project and run name separated by a slash:
"project/run_name". If None, wandb logging is not used.
Expand Down
1 change: 1 addition & 0 deletions chgnet/utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
AverageMeter,
cuda_devices_sorted_by_free_mem,
determine_device,
gpu_devices_sorted_by_free_mem,
mae,
mkdir,
read_json,
Expand Down
87 changes: 68 additions & 19 deletions chgnet/utils/common_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,23 @@
import os

import numpy as np
import pynvml
import torch
from torch import Tensor


def _default_accelerator() -> str:
"""Return the name of the best available torch accelerator backend.

Returns:
str: "cuda", "xpu" or "cpu" depending on what torch can see.
"""
if torch.cuda.is_available():
return "cuda"
if getattr(torch, "xpu", None) is not None and torch.xpu.is_available():
return "xpu"
return "cpu"


def determine_device(
use_device: str | None = None,
*,
Expand All @@ -18,8 +30,10 @@ def determine_device(

Args:
use_device (str): User specify device name
check_cuda_mem (bool): Whether to return cuda with most available memory
Default = False
check_cuda_mem (bool): Whether to return the accelerator with the most
available memory. Applies to both CUDA and XPU devices. Falls back
to an unindexed device when free memory cannot be queried, see
`gpu_devices_sorted_by_free_mem`. Default = False

Returns:
device (str): device name to be passed to model.to(device)
Expand All @@ -28,34 +42,69 @@ def determine_device(
if use_device in {"mps", None} and torch.backends.mps.is_available():
device = "mps"
else:
device = use_device or ("cuda" if torch.cuda.is_available() else "cpu")
if device == "cuda" and check_cuda_mem:
device = f"cuda:{cuda_devices_sorted_by_free_mem()[-1]}"
device = use_device or _default_accelerator()
if check_cuda_mem and device in {"cuda", "xpu"}:
devices_by_mem = gpu_devices_sorted_by_free_mem(device_type=device)
if devices_by_mem:
device = f"{device}:{devices_by_mem[-1]}"

return device


def cuda_devices_sorted_by_free_mem() -> list[int]:
"""List available CUDA devices sorted by increasing available memory.
def gpu_devices_sorted_by_free_mem(device_type: str | None = None) -> list[int]:
"""List available GPU devices sorted by increasing available memory.

Free memory is queried through torch itself (`mem_get_info`), so no
vendor-specific management library is needed. Indices are torch device
indices, so they respect `CUDA_VISIBLE_DEVICES` / `ZE_AFFINITY_MASK` and
can be passed straight to `.to(device)`.

To get the device with the most free memory, use the last list item.

Backend caveats:

- CUDA: `torch.cuda.mem_get_info` calls `cudaMemGetInfo` and reports
device-wide free memory, matching what NVML reported.
- XPU: `torch.xpu.mem_get_info` requires torch >= 2.6 and reads the SYCL
`ext_intel_free_memory` device query. On some multi-tile Intel GPUs
that query is reported per card rather than per tile, so the ranking
can be coarse. Returned indices are always valid either way.

An empty list is returned when free memory cannot be queried, in which
case callers should fall back to an unindexed device.

Args:
device_type (str): "cuda" or "xpu". If None, the available accelerator
is auto-detected. Default = None

Returns:
list[int]: CUDA device numbers sorted by increasing free memory.
list[int]: GPU device indices sorted by increasing free memory.
"""
if not torch.cuda.is_available():
device_type = device_type or _default_accelerator()
backend = getattr(torch, device_type, None)

if backend is None or not backend.is_available():
return []
# mps has no mem_get_info, and torch only added it for xpu in 2.6
if not hasattr(backend, "mem_get_info"):
return []

free_memories = [
backend.mem_get_info(idx)[0] for idx in range(backend.device_count())
]
return sorted(range(len(free_memories)), key=lambda idx: free_memories[idx])


free_memories = []
pynvml.nvmlInit()
device_count = pynvml.nvmlDeviceGetCount()
for idx in range(device_count):
handle = pynvml.nvmlDeviceGetHandleByIndex(idx)
info = pynvml.nvmlDeviceGetMemoryInfo(handle)
free_memories.append(info.free)
pynvml.nvmlShutdown()
def cuda_devices_sorted_by_free_mem() -> list[int]:
"""List available CUDA devices sorted by increasing available memory.

Deprecated alias kept for backward compatibility, prefer
`gpu_devices_sorted_by_free_mem`.

return sorted(range(len(free_memories)), key=lambda x: free_memories[x])
Returns:
list[int]: CUDA device numbers sorted by increasing free memory.
"""
return gpu_devices_sorted_by_free_mem(device_type="cuda")


class AverageMeter:
Expand Down
1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ dependencies = [
"ase>=3.23.0",
"cython>=3",
"numpy>=1.26",
"nvidia-ml-py>=13.580.82",
"pymatgen>=2024.9.10",
"torch>=2.4.1",
"typing-extensions>=4.12",
Expand Down
11 changes: 10 additions & 1 deletion tests/test_relaxation.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,20 @@ def test_relaxation(
no_mps = pytest.mark.skipif(
not torch.backends.mps.is_available() or "CI" in os.environ, reason="No MPS device"
)
no_xpu = pytest.mark.skipif(
getattr(torch, "xpu", None) is None or not torch.xpu.is_available(),
reason="No XPU device",
)


@pytest.mark.parametrize(
"use_device",
["cpu", pytest.param("cuda", marks=no_cuda), pytest.param("mps", marks=no_mps)],
[
"cpu",
pytest.param("cuda", marks=no_cuda),
pytest.param("xpu", marks=no_xpu),
pytest.param("mps", marks=no_mps),
],
)
def test_structure_optimizer_passes_kwargs_to_model(use_device: str) -> None:
relaxer = StructOptimizer(use_device=use_device)
Expand Down
Loading
Loading