Repository navigation
Conversation
Transfers 62 commits of work from the benrich37 fork that were built on top of the pre-reorganization pymatgen jdftx module. History of individual changes is preserved in benrich37/pymatgen master branch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Import all JDFTx changes from benrich37/pymatgen fork master branch
Sync from main
…he first place instead of initializing as a list of None (causes a lot of annoying UserWarning's down the line)
… almost-good-enough version of "add-U" parameter to the jdftxinfile master format file
…ss treatment for now
… to allow functionality, tests to ensure tag works as expected, deleting old "deleteme" file
…n order-sensitive values, enforced ordered equality in AbstractTag.is_equal_to. Added AbstractTag.read_from_tokens which by default behaves the same as `read` but also returns the number of tokens read. For TagContainer, read_from_tokens now performs a read on provided tokens until an unreadable token is encountered. Changed `TagContainer.read` behavior within subtag loop for positional subtags (subtags without written tagnames) to now call `_read_positional_subtag` (which relies on `read_from_tokens` instead of regular `read` to allow for possibly reading and discarding more than one token for a subtag. Moved initial magnetic moments validations to the JDFTXInfile class. Added a `get_species_values_tag_container` convenience function for initializing "initial-magnetic-moments" and "initial-oxidation-state" as a regular TagContainer instead of the old two-purpose InitMagMomTag. Removed `InitMagMomTag` as it is no longer needed. Added docustrings to helper functions in io/jdftx/inputs.py missing docustrings.
…lues are caught. Structure of "add-U" tag changed to contain a single "U-channels" subtag, which contains the expected structure of the "add-U" tag - this is done to allow JDFTx's behavior of not allowing multiple add-U lines, but obviously accepting multiple species to have add-U values described in that one line Changes made to `TagContainer.read_from_tokens` to prevent possible subtag keys from being read as values.
…e removal_path with _get_removal_path, and call _remove_at_path for each removal_path to remove that bad tag/subtag - prevents attempting to remove duplicate paths, and prioritizes deeper subtags and later pop indices to protect from removing tag paths with parents already removed and to protect from index shifting.
…att-scale subtags being specified as IntTag's instead of FloatTag's, added tests for converting named lattices to pymatgen Lattice objects.
…uations Lines ending in "\" are now joined with the following line by a single re.sub in JDFTXInfile.from_str. The previous _gather_tags implementation is kept commented out in case the regex approach is not accepted. Added test_line_continuation to check whitespace/comment handling around continuations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… function, adding testing for selective_dynamics_site_prop_to_jdftx_interpretable
…behavior can be reproduced in a single line and is only called once. Added test to test_jdftxinfile/test_jdftxstructure_infile_site_properties_conversion to ensure behavior is kept the same
…anmaps` which cleaned the values well enough itself such that the function could be replaced by np.nan. Removed `io.jdftx.inputs.get_popmaps`, since JDFTXInfile's dict inheritance allows `get_nanmaps` to operate directly on a JDFTXInfile.
…r `next` iteration. Added test to `io.jdftx.test_jdftxinfile` to ensure label fallback system performs as expected. Removed _multi_has/getattr functions and testing as they are no longer being used.
…ure.get_str` in `io.jdftx.inputs`, adding test to `test_jdftxinfile` to ensure changes to unit-converting code still works as intended.
…ded testing to ensure specie names of pmg Structure objects are left intact in JDFTXStructure.structure
…ding test to ensure hyperplane data can be safely passed back and forth between JDFTXStructure and JDFTXInfile tag reading/writing
…ystems for constraint vectors, fixed constraints being lost when a JDFTXInfile was initialized from a structure with constraints encoded into site properties, protect. Added error raising for some constraints which JDFTx would reject. General fixes to old misconception that ion prints will always be homogeneous shaping (this assumption breaks for some special case constraint types).
…MD output + testing
…n in eV / A + testing
… from standard JDFTx units to A / fs + testing
…m on top of regular parsing to allow `JDFTXOutfile.to_jdftxinfile` to exactly reproduce the ion coordinates without risk of rounding. On top of this, structural tags are now stored to 15 decimal precision (previously 12) to match JDFTx's output. `to_jdftxinfile` now also retains the coords-type of the originating output file. Fixed bug where JDFTXInfile.strip_structure_tags stripped a non-existent "lattice-scale" instead of "latt-scale". Fixed `_infile_special_format_to_pmg_lattice` which previously used pymatgen's built in Lattice construction - there is unfortunately a mismatch between pymatgen and JDFTx, where pymatgen begins by orienting the third lattice vector along the z axis, while JDFTx begins by orienting the first lattice vector along the x axis. Fixed bug in `condense_jdftxinputs` where any vestigial structure tags in the jdftxinput argument would remain in the condensed JDFTXInfile object.
…pdating linting to match upstream
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The fix needs an atomate2 update (two atomate2 tests index ["fluid-cation"] as a dict), so it goes to the end of the split, after the branches that do not: misc_fixes <- A1 <- A <- B0 <- B <- C <- D. Taken out here, and put back unchanged by misc_fixes_D: - can_repeat=True on tau-core, fluid-cation, fluid-anion, dump-Eresolved-density, elec-ex-corr-compare and dump-fermi-density (jdftxinfile_master_format.py) - the fluid ions in _fill_default_values (inputs.py), which returns their value as a list; here it handles fluid-solvent only - tests: test_repeatable_commands, test_repeated_commands_in_jdftx_echo_are_all_read, test_repeatable_tags_match_jdftx, test_adding_infiles_appends_repeatable_commands, test_fluid_ion_echo_is_read_and_written_back, the elec-ex-corr-compare case of test_hse06_is_accepted_with_a_warning, and the list form of two expected values in inputs_test_utils.py Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Conflict in tests/io/jdftx/test_jdftxinfile.py (adjacent import lines): kept this branch's JDFTXOutfile import and took misc_fixes_A's import line without MASTER_TAG_LIST. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
test_JDFTXOutfile_skimming failed on Windows CI with `assert 0.046875 < 0.046875`: time.process_time advances in steps of 1/64 s there, about the CPU time of one parse of these files, so a skimmed and a full parse could read the same. Each CPU time can be off by one step, so a speedup is now only tested where it is at least two steps: - Only the skims that leave out most of the file are timed: whole out file slices if there are several, otherwise geometry steps. The two geometry-only combinations of example_latmin.out are no longer timed. - Each timing is of enough parses for the full parse to take 6 clock steps. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
3 tasks done
Member
|
Automated PR review generated by Claude (posted from @shyuep's account). Based on code and CI status only, nothing was run. Please sanity-check before acting. CI: all 5 checks green on 4002007. Reviewed this PR's own commit (cc52f97: Good:
Substantive:
Nits:
|
… time The number of parses per timing was estimated from the wall time of one parse, while the times compared are CPU times. Wall time can exceed CPU time under load, which leaves the full parse spanning fewer than the intended 6 steps of a coarse CPU clock (1 step in a run with a simulated 1/64 s clock and the cores oversubscribed 4x). The estimate is now only the starting count: it is doubled until the measured CPU time of the full parse reaches 6 steps. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…th a warning A dict stored before velocities and constraints were kept in Cartesian angstrom (and fs) holds them as its ion lines had them, in coordinates it doesn't record. Such a dict has no movescale key, so from_dict now reads only what it read then (the structure and selective_dynamics) and warns about what it leaves out, instead of reading the old values as Cartesian angstrom/fs. Also adds a TODO for decoding arrays stored with dumpfn/MontyEncoder. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
3 tasks done
Contributor
Author
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[breaking] JDFTx IO Module Overhaul pt 5 - Round-trip serialization of parsed outputs
[breaking] JDFTx IO (5/6) - round-trip serialization of parsed outputs
Summary
Merge order: this is the fifth of the six PRs that #317 is split into (
misc_fixes_C).misc_fixes_A1,misc_fixes_A,misc_fixes_B0andmisc_fixes_Bneed to be merged first, in that order, andmisc_fixes_Dshould be merged after it.This PR makes parsed JDFTx outputs serializable: every output class can be dumped to JSON and loaded back unchanged. Until the PRs before it are merged, its diff also shows their changes.
Note for reviewers:
- The atomate2 change (materialsproject/atomate2#1659) needs to merge first. atomate2's JDFTx
CalculationOutput.parametersstoresJDFTXOutfile.as_dict()in a plaindict, and now that the dict has@module/@class, jobflow decodes it back into aJDFTXOutfile, which pydantic rejects for that field. That breaks every flow that chains atomate2 JDFTx jobs. The atomate2 change works with both old and new pymatgen-core, so it can merge first. There's a comment above the keys inJDFTXOutfile.as_dictto delete once it's in.Major changes:
Outputs - serialization
dumpfn/loadfn, jobflow'sjsanitizeand pydantic models, and passMatSciTest.assert_msonable-- Decoding used to raise
KeyError: 'lattice', and vibrations runs with imaginary frequencies couldn't be dumped to JSON at all--
JElStep,JElSteps,JOutStructures,JDFTXOutfileSliceandJDFTXOutfileare now subclasses ofMSONable, each with afrom_dict-- Every
as_dicthas@module/@classand returns plain JSON types, with numpy arrays as monty's{"@module": "numpy", "@class": "array", ...}dicts (asComputedEntry.as_dictdoes), so they load back as arrays--
JOutStructure.as_dict()is aStructuredict (lattice,sites, ...) plus the step data, and takesStructure.as_dict'sverbosity/fmtarguments--
JDFTXOutfileSlicegained dataclass fieldsinfile,geom_opt_label,ecomponentsand the AIMD fields (pe,ke,t_k,p_bar,tmd_fs,thermostat_velocity), soas_dictkeeps them--
JOutStructuregained annotated attributespe,ke,t_k,p_bar,tmd_fs,igp_forces(default None), so non-MD steps give None instead of raisingAttributeError-- Complex vibrational frequencies are stored as
[real, imag], and lattices read from out-file steps are C-ordered arrays-- With pydantic's default
ser_json_inf_nan="null", nan data values (p_bar, diverged electronic steps) come back as None; useser_json_inf_nan="constants"for exact round tripsTests
test_jdftx_out_filesis checked for an MSONable round trip, and an MD out file for thejsanitizeand pydantic round tripsChecklist
uv run pytest tests/core -k latticefor a lattice change — the full suite runs in CI)
pytest tests/io/jdftx: 603 passeduv run ruff check . && uv run ruff format --check .src/pymatgen/io/jdftxalso passes, and mypy reports no error in a JDFTx fileBreaking changes?
behavior changes, changed defaults/output, dropped Python versions, ...).
If so, prefix the title with
[breaking]and describe the change andmigration steps under
## Breaking Changesbelow.Breaking Changes
All of these are in
pymatgen.io.jdftx.atomate2: atomate2's JDFTx workflows need the atomate2 change (materialsproject/atomate2#1659), which drops the top-level
@module/@classkeys fromCalculationOutput.parameters. Without it, any flow that passes an atomate2 JDFTx job's output to a later job fails in pydantic validation. Reading stored task documents without decoding them still works.Changed serialized outputs (
as_dict):JElStep,JElSteps,JOutStructures,JDFTXOutfileSliceandJDFTXOutfiledicts have@module/@classand decode back to their classes withMontyDecoder/loadfn. Numpy arrays are monty array dicts.JOutStructure.as_dict()is aStructuredict (lattice,sites, ...) plus the step data, instead of its internal attributes (_lattice,_sites, ...).[real, imag].Changed outputs:
Todos
@module/@classkeys inJDFTXOutfile.as_dictas_dictencodes nested output objects once per nesting level (_json_dictruns in each class'sas_dict), which makes it about 2x slower than needed; encode once in theas_dictthat was called instead (see the TODO in_json_dict)