Skip to content

[breaking] JDFTx IO Module Overhaul pt 2 - input tags, JDFTXInfile and JDFTXStructure - #320

Open
benrich37 wants to merge 106 commits into
materialsproject:mainfrom
benrich37:misc_fixes_A
Open

benrich37 wants to merge 106 commits into
materialsproject:mainfrom
benrich37:misc_fixes_A

Conversation

@benrich37

@benrich37 benrich37 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

[breaking] JDFTx IO Module Overhaul pt 2 - input tags, JDFTXInfile and JDFTXStructure

Summary

Merge order: this is the second of the six PRs that #317 is split into (misc_fixes_A). misc_fixes_A1 needs to be merged first, and misc_fixes_B0, misc_fixes_B, misc_fixes_C and misc_fixes_D should be merged after it, in that order.

This PR holds the inputs side of #317: the rest of the tag definition fixes, the reworked tag classes, and the changes to JDFTXInfile and JDFTXStructure. The outputs modules aren't touched and keep working with it. Until misc_fixes_A1 is merged, the diff of this PR also shows its changes.

Note for reviewers:

- This PR may contain something to be considered arbitrary code execution The body of JDFTXInfile._get_lines in this PR has been shrunk to a single line using re.sub. Please let me know if this is acceptable or not. The old body of this function and its helper are no longer kept commented out in the code, but I can restore them if the current version is not okay.

- The out-file tests of these fixes are in misc_fixes_B, since they need its parsing fixes. This PR tests them on input files.

Major changes:

Inputs - tag definitions (jdftxinfile_master_format.py, jdftxinfile_ref_options.py)

  • feature 1: New commands and options
    -- add-U (stored under a single repeatable U-channels subtag, since JDFTx takes only one add-U line holding every species)
    -- initial-oxidation-state, chargeball, setVDW, set-atomic-radius, metadynamics-bond
    -- vibrations subtags dumpK, iConfiguration, nConfigurations (JDFTx builds since 2026-08-27 echo the last two even when unset, so their vibrations out files failed to parse)
    -- phiCavity for pcm-params, center for ionic-gaussian-potential
    -- fluid-site-params components now supports all 21 options (was 11)
    -- hyb-HSE06 for elec-ex-corr/elec-ex-corr-compare
    --- Previously left out on purpose, but now gives a JDFTXOptionWarning when read
  • feature 2: elec-n-bands is now a TagContainer with an optional multiplier
    -- JDFTx since 6ffe5cf0 (Oct 2025) the elec-n-bands format was switched from elec-n-bands <n> to elec-n-bands <n> <multiplier>
  • feature 3: Boundaries implemented/corrected for the numeric subtags of pcm-params, fluid-solvent/fluid-cation/fluid-anion and fluid-site-params, taken from the JDFTx source (ie epsBulk > 1, was > 0)
  • fix 1: Subtags with the wrong data type
    -- ion's moveScale corrected from IntTag to FloatTag
    -- pcm-params's lMax and ionic-dynamics's chainLengthT/chainLengthP corrected from FloatTag to IntTag
  • fix 2: Tags that couldn't be read in JDFTx's input file echo
    -- fluid-site-params (now JDFTx's syntax, fluid-site-params <component> <siteName> <key> <value> ...)
    -- debug with several options on one line
    -- pcm-params as JDFTx echoes it (empty cavityFile)
    -- ClassicalDFT fluid-solvent/fluid-cation/fluid-anion (JDFTx echoes translation as translationMode)
    -- The fluid representation subtag is now a string option (MuEps/Pomega/PsiAlpha) instead of a container of float subtags
  • fix 3: density-of-states keeps the order of its subcommands, since JDFTx's Occupied/Complete/SpinProjected/SpinTotal flags apply to the weights after them
    -- ie JDFTx's own echo Complete Total Occupied Orbital H 1 s used to be written back with the Orbital column before both flags, and Total Occupied Total raised
  • fix 4: initial-magnetic-moments is read into per-species values (previously was a single purpose InitMagMomTag, which just kept the line as a string and compared equal to any other value)
    -- New JDFTXInfile.validate_initial_magnetic_moments checks it against spintype and the ion tags (not run automatically) to ensure the initial-magnetic-moments command is compatible. It rejects moments for no-spin and spin-orbit runs (following JDFTx's behavior) and compares species (case insensitive, also a JDFTx behavior)
    -- Old-format serialized values can still be parsed
  • fix 5: thermostat-velocity and barostat-velocity take any number of values, as JDFTx does (they were fixed at 3 and 9 values, which only fits the default chainLengthT/chainLengthP)
    -- Their values are now {"v": [...]} instead of {"v1": ..., "v2": ..., ...} (the old form is still accepted)

Inputs - tag machinery (generic_tags.py)

  • feature 1: SubcommandTagContainer for commands whose subcommands are ordered (for density-of-states)
  • feature 2: StrListTag for several string options on one line (for debug)
  • feature 3: New tag fields
    -- StrTag.option_warnings (to raise the new JDFTXOptionWarning), StrTag.empty_means_unset (for compatibility with input echos), StrTag.species_name (boolean to trigger some functionality to allow setting non-standard specie names like "Fe1" instead of "Fe" without breaking anything)
    -- TagContainer.subtag_aliases (ie JDFTx echoes translation of a ClassicalDFT fluid component as translationMode), TagContainer.bare_is_noop
    -- AbstractTag.ordered for repeatable tags whose order matters when comparing
  • feature 4: AbstractTag._read_from_tokens, so positional subtags can read more than one token
  • feature 5: warn_user argument for validate_value_type to prevent MultiformatTag from raising warnings to the user when deciding on a format type
  • fix 1: TagContainer.write writes subtags in definition order, not the dict's key order
    -- ie setting {"EcutRho": 100, "Ecut": 20} would write elec-cutoff 100.0 20.0 which JDFTx would parse as a 100 Ha planewave cutoff and a 20 Ha density cutoff
  • fix 2: FloatTag accepts ints (not bools) and converts them to floats
  • fix 3: Entries of a repeated MultiformatTag are each validated, compared and written with their own format (helpful for ion lines where only some ions have a special-format constraint)
  • fix 4: "No valid read format" errors list each format's reason (the reasons were collected but never shown before)
  • fix 5: Removed InitMagMomTag (see tag definitions fix 4)

Inputs - JDFTXInfile

  • feature 1: Default value filling for comparisons
    -- get_differing_tags_from (and so get_differing_tags, get_filtered_differing_tags and is_comparable_to) now fills missing tags and subtags with JDFTx's defaults, so an input compares equal to JDFTx's echo of it
    -- Covers the defaults that depend on other tags (fluid-minimize per fluid type, ion-width for PCMs, lcao-params with smearing, and fluid-solvent parameters per solvent and temperature), checked against 70 JDFTx dry runs
    -- This was the default value filling left as a todo in Inputs - JDFTx IO Module pymatgen#4189 and JDFTXInfile Comparison Methods pymatgen#4416
  • feature 2: ion accepts JDFTx species names (species named after pseudopotential files, ie Fe1/Fe2 for an antiferromagnet), ignoring case as JDFTx does (ie ion fe ... raised)
    -- The element is the element symbol the name starts with (limitation: a C pseudopotential named Cs1.upf reads as Cs)
    -- Structures carry the names in a new jdftx_species site property (only when some name isn't its element symbol, so existing structures are unchanged), and from_structure writes them back
    -- New helpers get_species_elements, get_species_names (in inputs) and element_from_species_name (in generic_tags)
  • feature 3: read_line convenience method for adding one line to a JDFTXInfile
  • feature 4: from_str(skip_invalid_tags=...), so one unreadable command gives a warning instead of an error (misc_fixes_B uses it for the input echoed in an out file)
  • feature 5: nan handling: clean_infile_of_nans and write_file(strip_nan=...)
    -- The deepest subtags and latest indices are removed first, so removing one nan can't shift or orphan another
  • fix 1: include now splices the included file's lines in place, like JDFTx, instead of dict.update-ing the included file over the main one
    -- Repeatable tags from both files are kept in order (the included file's ion lines replaced the main file's); a non-repeatable tag in both files raises (JDFTx refuses those inputs too; the included file's value used to win); cyclic includes raise ValueError (was unbounded recursion); nested includes are also looked up next to the main file
    -- A missing include file gives a warning and keeps the include line (it raised)
  • fix 2: copy() is a deep copy (and skips the value-boundary checks)
  • fix 3: jif1 + jif2 validates the result
  • fix 4: strip_structure_tags strips latt-scale (it stripped a non-existent "lattice-scale", so latt-scale got applied twice on restarts)
  • fix 5: condense_jdftxinputs replaces the structure tags already in the input instead of adding to them (it left the old ions next to the new ones when they differed)
  • fix 6: get_filtered_differing_tags re-includes a tag from ensure_include_tags even if it was excluded twice (named in exclude_tags and in an excluded category)
  • fix 7: Out-of-bounds values give a warning when an input is read (from_file, from_str, from_dict, read_line) instead of raising, so stored inputs and the inputs echoed in out files stay loadable
    -- The values are kept, but left out when the input is written
    -- validate_boundaries() still raises when called directly; its new warn_only argument gives the warning instead

Inputs - JDFTXStructure

  • feature 1: Velocities and constraints are carried between Structure site properties, JDFTXStructure and JDFTXInfile
    -- JDFTXInfile.from_structure takes movescale, velocities, constraint_types, constraint_vectors, hyperplane_group_names and ignore_site_properties
    -- fill_site_properties argument added to to_pmg_structure, to_jdftxstructure and from_jdftxinfile
    -- JDFTXStructure's initializer takes any of these it isn't given from the structure's site properties (selective_dynamics, movescale, velocities, constraint_types, constraint_vectors, group_names) and keeps the structure's other site properties (it kept only selective_dynamics)
    -- JDFTXStructure.from_dict can read them back
    -- Velocities and constraints become site properties only when they form regular arrays (ie not for constraints on only some ions), so the structure still converts to ASE Atoms. Otherwise they are kept as attributes of the JDFTXStructure
  • feature 2: moveScale is kept as its own per-site value, so fractional values are no longer lost: new JDFTXStructure.movescale attribute, movescale site property and movescale argument of JDFTXInfile.from_structure
    -- selective_dynamics is kept in sync with it ([True, True, True] for a non-zero moveScale, [False, False, False] for 0)
    -- A site is fixed if either movescale or selective_dynamics marks it fixed, with a warning where they disagree. A movescale argument replaces the selective_dynamics site property
    -- A negative moveScale, or a movescale/selective_dynamics list of the wrong length, raises ValueError
    -- Selective dynamics that fix only some directions of a site raise NotImplementedError, since JDFTx's moveScale applies to every direction (such sites were frozen without a warning)
  • fix 1: Velocities are Cartesian Å/fs everywhere (was a confusing mess when mixed with direct coordinates before, this is the simplest fix)
  • fix 2: Named lattices (ie lattice Cubic 10, lattice Monoclinic ...)
    -- Interpret units as Bohr (JDFTx's length unit) instead of A and apply latt-scale
    -- Fix the explicit lattice construction to match JDFTx's GridInfo::setLatticeVectors convention (fix lattice "a" along the x axis) instead of pymatgen's (fix "c" along z axis) (would cause Monoclinic/Rhombohedral/Triclinic lattices to come out rotated)
  • fix 3: Constraints
    -- Constraints in ion lines are no longer dropped in to_jdftxstructure (it looked for a "constraint-type" key, the tag's is "constraint type") and never written back, and HyperPlane constraints raised
    -- They are now read, converted between lattice and Cartesian coordinates (a Linear direction is covariant, a Planar/HyperPlane normal contravariant, as in JDFTx), kept as site properties (see feature 1) and written back
    -- Constraints JDFTx would reject now raise (unknown type, missing or zero vector, a constraint on a fixed ion, several hyperplanes without group names)
  • fix 4: Structural tags are written with 15 decimals (was 12), matching what JDFTx prints
  • fix 5: sort_structure=True sorts the sites into the order JDFTx prints them in, and the per-site data (selective_dynamics, movescale, velocities, constraints) with them
    -- The per-site data wasn't sorted with the sites, so ie the moveScale flags ended up on the wrong ions
    -- JDFTx groups the ions by species, in the order the species first appear (ie O, H, O is printed as O, O, H). The sort was by electronegativity (H, O, O), so the sorted structure didn't line up with the out file
    -- Species are told apart by their names, ignoring case, so interleaved Fe1/Fe2 sites are grouped too
    -- The order can still differ from JDFTx's if the input names pseudopotential files with ion-species commands (instead of a $ID wildcard), since JDFTx then orders the species as those commands do (noted in the docstrings, see Todos)
  • fix 6: Removed a no-op quadratic loop in __post_init__

Removals

  • Removed InitMagMomTag (see tag definitions fix 4)

Tests and test files

  • 219 -> 458 JDFTx tests
  • New input file fixture test_jdftx_in_files/aimd.in
  • Removed the tests of the private helpers _multi_hasattr/_multi_getattr, which are gone

Checklist

  • Tests for the affected code pass locally (e.g. uv run pytest tests/core -k lattice
    for a lattice change — the full suite runs in CI)
    • pytest tests/io/jdftx: 458 passed
  • Lint passes: uv run ruff check . && uv run ruff format --check .
    • pyright on src/pymatgen/io/jdftx also passes, and mypy reports no error in a JDFTx file

Breaking changes?

  • This PR contains breaking changes (removals/renames, signature or
    behavior changes, changed defaults/output, dropped Python versions, ...).
    If so, prefix the title with [breaking] and describe the change and
    migration steps under ## Breaking Changes below.

Breaking Changes

All of these are in pymatgen.io.jdftx.

Removed:

  • generic_tags.InitMagMomTag. initial-magnetic-moments is now a TagContainer, so its value is a dict ({"species-values": [{"species-id": "Fe", "values": [...]}, ...]}) instead of a string.

Changed JDFTXInfile values (old forms that still work are noted):

  • density-of-states is a list of single-key dicts in order ([{"Etol": 1e-6}, {"Complete": True}, {"Total": True}, {"Orbital": {...}}]) instead of a dict. The old dict form (including old serialized JDFTXInfiles) is still accepted and read in dict order.
  • elec-n-bands is a dict ({"n": 14}, optionally with "multiplier") instead of an int. Setting an int still works.
  • The fluid component representation subtag is a string ("MuEps", "Pomega" or "PsiAlpha").
  • Int/float types now follow JDFTx. ion moveScale is now a float, and ion lines are written with it (1.0 instead of 1). pcm-params lMax and ionic-dynamics chainLengthT/chainLengthP are now ints. Files with the old forms still read.
  • fluid-site-params is written and read in JDFTx's syntax (fluid-site-params <component> <siteName> <key> <value> ...). The old pymatgen form (component H2O siteName O ...), which JDFTx doesn't accept, now raises. The dict keys are unchanged.
  • thermostat-velocity and barostat-velocity are {"v": [...]} with any number of values, instead of {"v1": ..., "v2": ..., ...}. Setting the old form still works.

Changed JDFTXInfile behavior:

  • Reading an input with an out-of-bounds value (from_file, from_str, from_dict, read_line) gives a warning instead of raising ValueError. The value is kept, and left out when the input is written. The check now also covers the new bounds (ie epsBulk 1, rhoMin 0). For the error, call validate_boundaries() on the result. Pass validate_value_boundaries=False to skip the check.
  • include splices the included file in place instead of updating the main file with it. A non-repeatable tag set in both files raises ValueError, and so does a cyclic include. A missing include file gives a warning instead of raising.
  • hyb-HSE06 is accepted with a JDFTXOptionWarning instead of being rejected.
  • TagContainer.write writes subtags in definition order, so written lines can differ in subtag order from before.
  • Structural tags are written with 15 decimals (was 12).
  • jif1 + jif2 validates its result, and copy() is now a deep copy instead of shallow.

Changed JDFTXStructure and structures built from inputs:

  • Velocities are Cartesian Å/fs (they were kept as given in the ion line). Serialized JDFTXStructure dicts made before this change hold velocities as given in the ion line, so from_dict leaves them out of such a dict (as it did before) and now warns about it. Re-create them from the original files to keep the velocities.
  • Named lattices (lattice Cubic|Tetragonal|Orthorhombic|Hexagonal|Rhombohedral|Monoclinic|Triclinic ...) are converted from bohr to Å (the lengths were used as Å), latt-scale is applied to them, and they are oriented like JDFTx (first vector along x). Lattice matrices and Cartesian coordinates from such inputs change.
  • Constraints in ion lines are read (they were dropped) and kept, with velocities, as site properties of the structure when they form regular arrays (ie constraints on only some ions stay attributes of the JDFTXStructure). Constraint vectors are Cartesian. Pass fill_site_properties=False to leave these site properties out.
  • Structures built from inputs get a movescale site property next to selective_dynamics (also with fill_site_properties=False), and JDFTXStructure.selective_dynamics is a list of per-direction booleans ([[True, True, True], ...]) instead of one 0/1 value per site.
  • Selective dynamics that fix only some directions of a site (ie [True, True, False] from a POSCAR slab) raise NotImplementedError in JDFTXStructure and JDFTXInfile.from_structure (such sites were frozen without a warning). Pass selective_dynamics or movescale that fix or free each site entirely; a Linear or Planar constraint (constraint_types/constraint_vectors) can limit the directions a free site moves in.
  • A negative moveScale raises ValueError (JDFTx rejects it too).
  • JDFTXStructure keeps the site properties of the structure it is given (it kept only selective_dynamics).
  • sort_structure=True sorts the sites into JDFTx's order (grouped by species name, in the order the species first appear) instead of by electronegativity, so the site order it gives changes (ie O, H, O gives O, O, H, was H, O, O). Each site's moveScale, velocity and constraint now move with it (they stayed in the unsorted order). For the old order, sort the structure with get_sorted_structure() yourself instead of passing sort_structure=True.
  • ion species names that aren't element symbols (ie Fe1, or fe) now parse, and structures get a jdftx_species site property when a name isn't its element symbol.
  • Lattices built from input files are C-ordered, so Cartesian coordinates can differ from before in the last digit.

Changed stored data:

  • Code that queries stored atomate2 task documents will see calc_inputs.jdftxinfile elec-n-bands as {"n": 14} and initial-magnetic-moments as {"species-values": [...]}, and selective_dynamics and movescale on input structures.

Todos

  • sort_structure doesn't follow the species order set by explicit ion-species <file> commands (see JDFTXStructure fix 5)
  • JDFTXStructure velocities don't survive dumpfn/loadfn: they load as monty array dicts, which get_str fails on (before this PR they were dropped); decode them in from_dict (see the TODO there). Storing with jsanitize, as jobflow does, works
  • Sites fixed along only some directions raise NotImplementedError for now; they could be written with a Linear or Planar constraint instead (see JDFTXStructure feature 2)

benrich37 and others added 30 commits April 9, 2026 11:31
Transfers 62 commits of work from the benrich37 fork that were built
on top of the pre-reorganization pymatgen jdftx module. History of
individual changes is preserved in benrich37/pymatgen master branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Import all JDFTx changes from benrich37/pymatgen fork master branch
…he first place instead of initializing as a list of None (causes a lot of annoying UserWarning's down the line)
… almost-good-enough version of "add-U" parameter to the jdftxinfile master format file
… to allow functionality, tests to ensure tag works as expected, deleting old "deleteme" file
…n order-sensitive values, enforced ordered equality in AbstractTag.is_equal_to. Added AbstractTag.read_from_tokens which by default behaves the same as `read` but also returns the number of tokens read. For TagContainer, read_from_tokens now performs a read on provided tokens until an unreadable token is encountered. Changed `TagContainer.read` behavior within subtag loop for positional subtags (subtags without written tagnames) to now call `_read_positional_subtag` (which relies on `read_from_tokens` instead of regular `read` to allow for possibly reading and discarding more than one token for a subtag. Moved initial magnetic moments validations to the JDFTXInfile class. Added a `get_species_values_tag_container` convenience function for initializing "initial-magnetic-moments" and "initial-oxidation-state" as a regular TagContainer instead of the old two-purpose InitMagMomTag. Removed `InitMagMomTag` as it is no longer needed. Added docustrings to helper functions in io/jdftx/inputs.py missing docustrings.
…lues are caught. Structure of "add-U" tag changed to contain a single "U-channels" subtag, which contains the expected structure of the "add-U" tag - this is done to allow JDFTx's behavior of not allowing multiple add-U lines, but obviously accepting multiple species to have add-U values described in that one line Changes made to `TagContainer.read_from_tokens` to prevent possible subtag keys from being read as values.
…e removal_path with _get_removal_path, and call _remove_at_path for each removal_path to remove that bad tag/subtag - prevents attempting to remove duplicate paths, and prioritizes deeper subtags and later pop indices to protect from removing tag paths with parents already removed and to protect from index shifting.
…att-scale subtags being specified as IntTag's instead of FloatTag's, added tests for converting named lattices to pymatgen Lattice objects.
…uations

Lines ending in "\" are now joined with the following line by a single re.sub in JDFTXInfile.from_str. The previous
_gather_tags implementation is kept commented out in case the regex approach is not accepted. Added
test_line_continuation to check whitespace/comment handling around continuations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… function, adding testing for selective_dynamics_site_prop_to_jdftx_interpretable
…behavior can be reproduced in a single line and is only called once. Added test to test_jdftxinfile/test_jdftxstructure_infile_site_properties_conversion to ensure behavior is kept the same
…anmaps` which cleaned the values well enough itself such that the function could be replaced by np.nan. Removed `io.jdftx.inputs.get_popmaps`, since JDFTXInfile's dict inheritance allows `get_nanmaps` to operate directly on a JDFTXInfile.
…r `next` iteration. Added test to `io.jdftx.test_jdftxinfile` to ensure label fallback system performs as expected. Removed _multi_has/getattr functions and testing as they are no longer being used.
…ure.get_str` in `io.jdftx.inputs`, adding test to `test_jdftxinfile` to ensure changes to unit-converting code still works as intended.
…ded testing to ensure specie names of pmg Structure objects are left intact in JDFTXStructure.structure
…ding test to ensure hyperplane data can be safely passed back and forth between JDFTXStructure and JDFTXInfile tag reading/writing
…ystems for constraint vectors, fixed constraints being lost when a JDFTXInfile was initialized from a structure with constraints encoded into site properties, protect. Added error raising for some constraints which JDFTx would reject. General fixes to old misconception that ion prints will always be homogeneous shaping (this assumption breaks for some special case constraint types).
… from standard JDFTx units to A / fs + testing
…m on top of regular parsing to allow `JDFTXOutfile.to_jdftxinfile` to exactly reproduce the ion coordinates without risk of rounding. On top of this, structural tags are now stored to 15 decimal precision (previously 12) to match JDFTx's output. `to_jdftxinfile` now also retains the coords-type of the originating output file. Fixed bug where JDFTXInfile.strip_structure_tags stripped a non-existent "lattice-scale" instead of "latt-scale". Fixed `_infile_special_format_to_pmg_lattice` which previously used pymatgen's built in Lattice construction - there is unfortunately a mismatch between pymatgen and JDFTx, where pymatgen begins by orienting the third lattice vector along the z axis, while JDFTx begins by orienting the first lattice vector along the x axis. Fixed bug in `condense_jdftxinputs` where any vestigial structure tags in the jdftxinput argument would remain in the condensed JDFTXInfile object.
benrich37 and others added 22 commits November 17, 2026 11:56
…_parse_emin_lines` into a ValueError to allow the higher up error catchers to catch it instead of aborting the whole parse
…r slices that failed to parse and raise a ValueError if ALL slices failed to parse
…t options to avoid issues with exact-case matching checks down the line
… to set up ASE in testing environment I moved this all to pymatgen-core, so when the homogeneity protections got removed the tests that should have failed were not being ran). Adding a `_sync_movescale_and_selective_dynamics` to be called in the `JDFTXStructure` post-init to ensure movescales and selective dynamics agree with each other (if a movescale > 0, sd should be [1,1,1], else [0,0,0]). Remove unnecessary conversions of movescales to selective dynamics. Make "movescale" just as easy to set as a "selective_dynamics" (since they both encode overlapping but also unique information)
…t very complicated, so to err on the side simplicity fixing an atom in place with either movescale or selective dynamics os absolute
…moving the now-unused JDFTXOutfileSlice._get_initial_structure, deleting commented out code
…. To block bad in files with errors, run JDFTXInfile.validate_boundaries() directly without the warn_only=True kwarg
…of misc_fixes)

First branch of the misc_fixes split (misc_fixes <- A1 <- A <- B0 <- B <- C), built on the
head of misc_fixes (73f2d02). This commit removes every JDFTx change of
misc_fixes except the fixes listed below, so the branch's diff against
upstream/main is A1 alone. The branches above re-add the rest, one division
each, and misc_fixes_C ends at the tree of misc_fixes again.

What the branch keeps: tag-table and generic-tag fixes that work on
upstream/main alone, with their input-level tests. Every line is in its final
misc_fixes form.

- electric-field / perturb-electric-field Ex/Ey/Ez are FloatTags (were IntTags)
- latt-scale s0/s1/s2 are FloatTags (were IntTags)
- perturb-incommensurate-wavefunctions EcutOld is a FloatTag
- rhoExternal includeSelfEnergy is a yes/no BoolTag (was a FloatTag)
- Vexternal filename subtags use write_tagname=False, so both forms parse
- van-der-waals: D3BJ option; a bare van-der-waals reads as 0.0
  (new FloatTag.value_if_bare)
- charged-defect-correction is a MultiformatTag of its three JDFTx forms
- perturb-minimize algorithm options are CG|MINRES (was MINRES|CGIMINRES)
- fluid-solvent: CHCl3 option
- dump: HC variable
- IntTags for electron-scattering computeRange iqStart/iqStop (lb=1),
  density-of-states Slice/AtomSlice Miller indices, and the fluid
  quad_nAlpha/quad_nBeta/quad_nGamma (with JDFTx's bounds)
- MultiformatTag.validate_value_bounds checks the bounds of the format the
  value matches (each entry of a repeated tag on its own); a nan passes any
  bounds check
- StrTag options match ignoring case, and read as the option's own spelling

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Second PR of the misc_fixes split (misc_fixes <- A1 <- A <- B0 <- B <- C). This commit
re-adds the inputs side that A1's commit stripped. It changes only the
inputs side of pymatgen.io.jdftx; the outputs modules are still upstream's
and keep working with it.

Tag table and machinery (jdftxinfile_master_format.py,
jdftxinfile_ref_options.py, generic_tags.py), on top of A1:
- new commands and options (add-U, initial-oxidation-state, chargeball,
  setVDW, set-atomic-radius, metadynamics-bond, vibrations iConfiguration /
  nConfigurations, hyb-HSE06 with a JDFTXOptionWarning, ...)
- elec-n-bands as a TagContainer with an optional multiplier
- JDFTx's bounds on fluid, pcm-params, electron-scattering and quad subtags
- repeatable tau-core, fluid-cation/anion, dump-Eresolved-density,
  elec-ex-corr-compare, dump-fermi-density
- ion moveScale as a float; thermostat/barostat-velocity of any chain length
- fluid-site-params, debug, pcm-params, ClassicalDFT fluid echoes readable;
  density-of-states keeps its subcommand order
- SubcommandTagContainer, StrListTag, positional multi-token subtags, new
  tag fields; InitMagMomTag removed (initial-magnetic-moments is a
  TagContainer, validated by JDFTXInfile)

JDFTXInfile (inputs.py, jdftxinfile_default_inputs.py, sets.py):
- default value filling for comparisons with JDFTx's echo
- JDFTx species names (jdftx_species site property)
- read_line, from_str(skip_invalid_tags=...), nan handling
- include spliced in place; deep copy(); `+` appends repeatable tags
- out-of-bounds values warn when read; validate_boundaries() raises
- fixes to strip_structure_tags, condense_jdftxinputs,
  get_filtered_differing_tags, from_structure(ignore_site_properties=True)

JDFTXStructure (inputs.py):
- velocities in Cartesian A/fs and constraint vectors carried between
  Structure, JDFTXStructure and JDFTXInfile
- named lattices built as JDFTx does, with latt-scale, C-ordered
- movescale and selective dynamics kept in sync (partial selective
  dynamics raises NotImplementedError); ions grouped like JDFTx
- structural tags written with 15 decimals

Tests: input-level tests for all of the above, new fixture
test_jdftx_in_files/aimd.in. The out-file regression tests for these fixes
and test_get_differing_tags_fills_defaults need the outputs PR (B) and
ship there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The fix needs an atomate2 update (two atomate2 tests index ["fluid-cation"] as a dict), so it goes to the end of the
split, after the branches that do not: misc_fixes <- A1 <- A <- B0 <- B <- C <- D.

Taken out here, and put back unchanged by misc_fixes_D:
- can_repeat=True on tau-core, fluid-cation, fluid-anion, dump-Eresolved-density, elec-ex-corr-compare and
  dump-fermi-density (jdftxinfile_master_format.py)
- the fluid ions in _fill_default_values (inputs.py), which returns their value as a list; here it handles
  fluid-solvent only
- tests: test_repeatable_commands, test_repeated_commands_in_jdftx_echo_are_all_read,
  test_repeatable_tags_match_jdftx, test_adding_infiles_appends_repeatable_commands,
  test_fluid_ion_echo_is_read_and_written_back, the elec-ex-corr-compare case of
  test_hse06_is_accepted_with_a_warning, and the list form of two expected values in inputs_test_utils.py

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@benrich37
benrich37 marked this pull request as ready for review November 19, 2026 21:12
@benrich37
benrich37 requested a review from shyuep as a code owner November 19, 2026 21:12
@shyuep

shyuep commented Nov 20, 2026

Copy link
Copy Markdown
Member

Automated PR review generated by Claude (posted from @shyuep's account). Based on code and CI status only, nothing was run. Please sanity-check before acting.

CI: all 5 checks green. Scope: the diff includes #319. This PR's own increment (b53e226 plus the move of the repeatable-tags fix to #324, ~325 kB) is mostly the inputs half of #317, which was reviewed there twice. This review covers the description, generic_tags.py and the JDFTXStructure/_fill_default_values paths. It does not cover every hunk.

Good:

  • The Breaking Changes section is complete and gives a migration for each item: old density-of-states dicts, int elec-n-bands, and {"v1": ...} velocities still load.
  • Named lattices are now read in bohr with latt-scale applied and oriented like GridInfo::setLatticeVectors. That fixes silently wrong cells for Monoclinic/Rhombohedral/Triclinic inputs.
  • Per-site data (moveScale, velocities, constraints) now moves with the sites under sort_structure=True. Previously moveScale flags could land on the wrong ions.
  • Warn-on-read with validate_boundaries() still raising answers the earlier concern on JDFTx IO module overhaul #317.

Substantive:

  1. Silent reinterpretation of stored velocities. The description notes that serialized JDFTXStructure dicts from before this change hold velocities in the old form, and that from_dict now reads them as Cartesian Å/fs without a warning. That is a silent data-correctness change for anything already stored (atomate2 task documents, MD restarts). Add a format marker to JDFTXStructure.as_dict (e.g. "velocity_units": "cartesian_A_per_fs") and, in from_dict, warn or refuse when velocities are present without it. A sentence in the description is not enough for stored data.
  2. Old stored initial-magnetic-moments. The tag is now a TagContainer ({"species-values": [...]}). The description lists compatibility paths for density-of-states, elec-n-bands and the velocities, but not for this one. Is a stored JDFTXInfile dict with the old string value still accepted by from_dict? If not, it should be, or it should go under Breaking Changes with the atomate2 impact.
  3. Merge order. This PR on its own changes the stored-document shapes listed under "Changed stored data" (elec-n-bands, initial-magnetic-moments). Confirm whether atomate2 needs a matching change for this part, as [breaking] JDFTx IO Module Overhaul pt 5 - Round-trip serialization of parsed outputs #323 and [breaking] JDFTx IO Module Overhaul pt 6 - repeatable input tags #324 do, and link it if so.

Nits:

  • Removing generic_tags.InitMagMomTag outright is acceptable for a [breaking] PR, but a module-level __getattr__ that raises with a pointer to the new TagContainer would make the failure self-explanatory.

…th a warning

A dict stored before velocities and constraints were kept in Cartesian angstrom (and fs) holds them as its ion
lines had them, in coordinates it doesn't record. Such a dict has no movescale key, so from_dict now reads only
what it read then (the structure and selective_dynamics) and warns about what it leaves out, instead of reading
the old values as Cartesian angstrom/fs.

Also adds a TODO for decoding arrays stored with dumpfn/MontyEncoder.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@benrich37

Copy link
Copy Markdown
Contributor Author
  • Substantive 1 (stored velocities): Fixed in 4863a6e, without a new key. JDFTXStructure.as_dict never wrote movescale before this PR and always writes it now, so a dict without that key is an old one. For those, from_dict now reads only what it read before (the structure and selective_dynamics) and warns that it is leaving out the stored velocities and constraints, instead of reading them as Cartesian Å/fs. The old from_dict ignored them too, so nothing that used to load is lost. The description is updated to match. (atomate2 doesn't use JDFTXStructure, so its task documents aren't affected.)
  • Substantive 2 (old initial-magnetic-moments): Yes, a stored string value still loads. JDFTXInfile.from_dict({"initial-magnetic-moments": "Fe 1 2 3 Ni 0"}) gives {"species-values": [{"species-id": "Fe", "values": [1.0, 2.0, 3.0]}, {"species-id": "Ni", "values": [0.0]}]}.
  • Substantive 3 (atomate2): No atomate2 change is needed for this part: atomate2's JDFTx tests pass unchanged against this branch. Only code that queries stored task documents sees the new shapes, which is what "Changed stored data" lists. The atomate2 change (Compatibility with incoming update to JDFTx pymatgen-core IO module atomate2#1659) is only needed for [breaking] JDFTx IO Module Overhaul pt 5 - Round-trip serialization of parsed outputs #323 and [breaking] JDFTx IO Module Overhaul pt 6 - repeatable input tags #324.
  • Nit (InitMagMomTag): Leaving the removal as is, it is listed under Breaking Changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants