hydrodynamics + gravity

Castro's hydrodynamics is based on a compressible flow formulation in Eulerian coordinates. Castro uses a dimensionally-unsplit 2nd order Godunov method for construct the fluxes through the interfaces of each cell (a choice of piecewise linear or piecewise parabolic reconstruction + characteristic tracing is provided). The hydrodynamics is designed to work with a general (convex) equation of state, and a variety of Riemann solvers are provided.

Newtonian gravitational forces are supported through either a simple monopole approximation or the full Poisson equation using geometry multigrid. Isolated boundary conditions for the Poisson solver allow for modeling complex mass distributions. Several implementations for incorporating gravity into the energy equation are implemented, including an explicit source term and a conservative flux formulation.


Radiation support includes gray or multigroup flux limited diffusion in a mixed-frame or comoving frame, including all coupling terms to O(v/c). The radiation update is done implicitly using the linear solvers provided by the hypre package.


A default γ-law equation of state and general-composition non-reacting network as distributed in the main Castro git repo. Additional equations of state (including a general stellar EOS) and many nuclear reaction networks are distributed in the Microphysics git repo. Maintaining them in a separate repo allows all of the AMReX astrophysics codes share the same microphysics.


All Castro plotfiles store information about the build environment (build machine, build date, compiler version and flags), run environment (output date, output dir, number of processors, wall clock time), and code versions used (git hashes for the main Castro source and AMReX source, and support repos, if available). This allows for a recovery of the code base used for previous results.

Castro is run through a nightly regression test suite that checks the output, bit-for-bit, against stored benchmarks for many of the problem setups (including real science runs).

adaptive mesh refinement

The approach to AMR in Castro uses a nested hierarchy of logically-rectangular grids with simultaneous refinement of the grids in both space and time. The integration algorithm on the grid hierarchy is a recursive procedure in which coarse grids are advanced in time, fine grids are advanced multiple steps to reach the same time as the coarse grids and the data at different levels are then synchronized. A separate synchronization is done for gravity.

AMReX grid showing 2 levels.

parallel performance

Castro uses a hybrid programming model based on MPI and OpenMP. Distribution of grid patches to nodes using MPI provides a natural coarse-grained approach to distributing the computational work, while threading of loops over zones in a grid using OpenMP provides effective fine-grained parallelization. Castro uses a new tiled approach to OpenMP that can effectively take advantage of manycore processors like the Intel Phi's.

Castro strong scaling for a pure hydro + real EOS Sedov problem on the OLCF titan machine.

Castro strong scaling for the WD merger problem with full Poisson gravity on the OLCF titan machine.