Merge branch 'main' of github.com:PhasicFlow/phasicFlow

This commit is contained in:
Hamidreza 2025-03-09 21:14:02 +03:30
commit 389e42ee1f
18 changed files with 425 additions and 396 deletions

View File

@ -1,39 +1,50 @@
<div align="center">
<img src="doc/phasicFlow_logo_github.png" style="width: 400px;">
<img src="doc/phasicFlow_logo_github.png" style="width: 400px;" alt="PhasicFlow Logo">
</div>
## **PhasicFlow: High-Performance Discrete Element Method Simulations**
**PhasicFlow** is a parallel C++ code for performing DEM simulations. It can run on shared-memory multi-core computational units such as multi-core CPUs or GPUs (for now it works on CUDA-enabled GPUs). The parallelization method mainly relies on loop-level parallelization on a shared-memory computational unit. You can build and run PhasicFlow in serial mode on regular PCs, in parallel mode for multi-core CPUs, or build it for a GPU device to off-load computations to a GPU. In its current statues you can simulate millions of particles (up to 80M particles tested) on a single desktop computer. You can see the [performance tests of PhasicFlow](https://github.com/PhasicFlow/phasicFlow/wiki/Performance-of-phasicFlow) in the wiki page.
PhasicFlow is a robust, open-source C++ framework designed for the efficient simulation of granular materials using the Discrete Element Method (DEM). Leveraging parallel computing paradigms, PhasicFlow is capable of executing simulations on shared-memory multi-core architectures, including CPUs and NVIDIA GPUs (CUDA-enabled). The core parallelization strategy focuses on loop-level parallelism, enabling significant performance gains on modern hardware. Users can seamlessly transition between serial execution on standard PCs, parallel execution on multi-core CPUs (OpenMP), and accelerated simulations on GPUs. Currently, PhasicFlow supports simulations involving up to 80 million particles on a single desktop workstation. Detailed performance benchmarks are available on the [PhasicFlow Wiki](https://github.com/PhasicFlow/phasicFlow/wiki/Performance-of-phasicFlow).
**MPI** parallelization with dynamic load balancing is under development. With this level of parallelization, PhasicFlow can leverage the computational power of **multi-gpu** workstations or clusters with distributed memory CPUs.
In summary PhasicFlow can have 6 execution modes:
1. Serial on a single CPU core,
2. Parallel on a multi-core computer/node (using OpenMP),
3. Parallel on an nvidia-GPU (using Cuda),
4. Parallel on distributed memory workstation (Using MPI)
5. Parallel on distributed memory workstations with multi-core nodes (using MPI+OpenMP)
6. Parallel on workstations with multiple GPUs (using MPI+Cuda).
## How to build?
You can build PhasicFlow for CPU and GPU executions. The latest release of PhasicFlow is v-0.1. [Here is a complete step-by-step procedure for building phasicFlow-v-0.1.](https://github.com/PhasicFlow/phasicFlow/wiki/How-to-Build-PhasicFlow).
**Scalable Parallelism: MPI Integration**
## Online code documentation
You can find a full documentation of the code, its features, and other related materials on [online documentation of the code](https://phasicflow.github.io/phasicFlow/)
Ongoing development includes the integration of MPI-based parallelization with dynamic load balancing. This enhancement will extend PhasicFlow's capabilities to distributed memory environments, such as multi-GPU workstations and high-performance computing clusters. Upon completion, PhasicFlow will offer six distinct execution modes:
## How to use PhasicFlow?
You can navigate into [tutorials folder](./tutorials) in the phasicFlow folder to see some simulation case setups. If you need more detailed discription, visit our [wiki page tutorials](https://github.com/PhasicFlow/phasicFlow/wiki/Tutorials).
1. **Serial Execution:** Single-core CPU.
2. **Shared-Memory Parallelism:** Multi-core CPU (OpenMP).
3. **GPU Acceleration:** NVIDIA GPU (CUDA).
4. **Distributed-Memory Parallelism:** MPI.
5. **Hybrid Parallelism:** MPI + OpenMP.
6. **Multi-GPU Parallelism:** MPI + CUDA.
## [PhasicFlowPlus](https://github.com/PhasicFlow/PhasicFlowPlus)
PhasicFlowPlus is and extension to PhasicFlow for simulating particle-fluid systems using resolved and unresolved CFD-DEM. [See the repository of this package.](https://github.com/PhasicFlow/PhasicFlowPlus)
# **Build and Installation**
PhasicFlow can be compiled for both CPU and GPU execution.
* **Current Development (v-1.0):** Comprehensive build instructions are available [here](https://github.com/PhasicFlow/phasicFlow/wiki/How-to-build-PhasicFlow%E2%80%90v%E2%80%901.0).
* **Latest Release (v-0.1):** Detailed build instructions are available [here](https://github.com/PhasicFlow/phasicFlow/wiki/How-to-Build-PhasicFlow).
# **Comprehensive Documentation**
In-depth documentation, including code structure, features, and usage guidelines, is accessible via the [online documentation portal](https://phasicflow.github.io/phasicFlow/).
## **Tutorials and Examples**
Practical examples and simulation setups are provided in the [tutorials directory](./tutorials). For detailed explanations and step-by-step guides, please refer to the [tutorial section on the PhasicFlow Wiki](https://github.com/PhasicFlow/phasicFlow/wiki/Tutorials).
# **PhasicFlowPlus: Coupled CFD-DEM Simulations**
PhasicFlowPlus is an extension of PhasicFlow that facilitates the simulation of particle-fluid systems using resolved and unresolved CFD-DEM methods. The repository for PhasicFlowPlus can be found [here](https://github.com/PhasicFlow/PhasicFlowPlus).
## Supporting packages
* [Kokkos](https://github.com/kokkos/kokkos) from National Technology & Engineering Solutions of Sandia, LLC (NTESS)
* [CLI11 1.8](https://github.com/CLIUtils/CLI11) from University of Cincinnati.
# How to cite PhasicFlow?
## How to cite PhasicFlow
If you are using PhasicFlow in your research or industrial work, cite the following [article](https://www.sciencedirect.com/science/article/pii/S0010465523001662):
```
@article{NOROUZI2023108821,
@article
{
NOROUZI2023108821,
title = {PhasicFlow: A parallel, multi-architecture open-source code for DEM simulations},
journal = {Computer Physics Communications},
volume = {291},
@ -46,3 +57,10 @@ author = {H.R. Norouzi},
keywords = {Discrete element method, Parallel computing, CUDA, GPU, OpenMP, Granular flow}
}
```
# **Dependencies**
PhasicFlow relies on the following external libraries:
* **Kokkos:** A performance portability ecosystem developed by National Technology & Engineering Solutions of Sandia, LLC (NTESS). ([https://github.com/kokkos/kokkos](https://github.com/kokkos/kokkos))
* **CLI11 1.8:** A command-line interface parser developed by the University of Cincinnati. ([https://github.com/CLIUtils/CLI11](https://github.com/CLIUtils/CLI11))

View File

@ -4,56 +4,57 @@
\* ------------------------------------------------------------------------- */
objectName interaction;
objectType dicrionary;
objectType dictionary;
fileFormat ASCII;
/*---------------------------------------------------------------------------*/
materials (glassMat wallMat); // a list of materials names
densities (2500.0 2500); // density of materials [kg/m3]
contactListType sortedContactList;
model
{
contactForceModel nonLinearLimited;
rollingFrictionModel normal;
Yeff (1.0e6 1.0e6 // Young modulus [Pa]
1.0e6);
Geff (0.8e6 0.8e6 // Shear modulus [Pa]
0.8e6);
nu (0.25 0.25 // Poisson's ratio [-]
0.25);
en (0.97 0.85 // coefficient of normal restitution
1.00);
et (1.0 1.0 // coefficient of tangential restitution
1.0);
mu (0.65 0.65 // dynamic friction
0.65);
mur (0.1 0.1 // rolling friction
0.1);
}
contactSearch
{
method NBS;
wallMapping cellMapping;
NBSInfo
updateInterval 10;
sizeRatio 1.1;
cellExtent 0.55;
adjustableBox Yes;
}
model
{
updateFrequency 10; // each 20 timesteps, update neighbor list
sizeRatio 1.05; // bounding box size to particle diameter (max)
contactForceModel nonLinearLimited;
rollingFrictionModel normal;
/*
Property (glassMat-glassMat glassMat-wallMat
wallMat-wallMat);
*/
Yeff (1.0e6 1.0e6
1.0e6); // Young modulus [Pa]
Geff (0.8e6 0.8e6
0.8e6); // Shear modulus [Pa]
nu (0.25 0.25
0.25); // Poisson's ratio [-]
en (0.97 0.85
1.00); // coefficient of normal restitution
mu (0.65 0.65
0.65); // dynamic friction
mur (0.1 0.1
0.1); // rolling friction
}
cellMappingInfo
{
updateFrequency 10; // each 20 timesteps, update neighbor list
cellExtent 0.6; // bounding box for particle-wall search (> 0.5)
}
}

View File

@ -5,10 +5,8 @@
objectName particleInsertion;
objectType dicrionary;
active no; // is insertion active?
collisionCheck No; // not implemented for yes
fileFormat ASCII;
active No; // is insertion active?

View File

@ -2,14 +2,14 @@
| phasicFlow File |
| copyright: www.cemf.ir |
\* ------------------------------------------------------------------------- */
objectName sphereDict;
objectType sphereShape;
objectName shapes;
objectType dictionary;
fileFormat ASCII;
/*---------------------------------------------------------------------------*/
// names of shapes
names (smallSphere largeSphere);
// diameter of shapes (m)
diameters (0.004 0.005);
// material names for shapes
materials (lightMat heavyMat);
names (glassBead); // names of shapes
diameters (0.003); // diameter of shapes
materials (glassMat); // material names for shapes

View File

@ -1,11 +0,0 @@
/* -------------------------------*- C++ -*--------------------------------- *\
| phasicFlow File |
| copyright: www.cemf.ir |
\* ------------------------------------------------------------------------- */
objectName sphereDict;
objectType sphereShape;
names (glassBead); // names of shapes
diameters (0.003); // diameter of shapes
materials (glassMat); // material names for shapes

View File

@ -0,0 +1,46 @@
/* -------------------------------*- C++ -*--------------------------------- *\
| phasicFlow File |
| copyright: www.cemf.ir |
\* ------------------------------------------------------------------------- */
objectName domainDict;
objectType dictionary;
fileFormat ASCII;
/*---------------------------------------------------------------------------*/
globalBox // Simulation domain: every particles that goes outside this domain will be deleted
{
min (-0.2 -0.2 -0.0);
max ( 0.2 0.2 1.6);
}
boundaries
{
left
{
type exit; // other options: periodic, reflective
}
right
{
type exit; // other options: periodic, reflective
}
bottom
{
type exit; // other options: periodic, reflective
}
top
{
type exit; // other options: periodic, reflective
}
rear
{
type exit; // other options: periodic, reflective
}
front
{
type exit; // other options: periodic, reflective
}
}

View File

@ -5,59 +5,82 @@
objectName geometryDict;
objectType dictionary;
fileFormat ASCII;
motionModel rotatingAxisMotion;
motionModel rotatingAxis; // motion model: rotating object around an axis
rotatingAxisInfo // information for rotatingAxisMotion motion model
{
rotAxis
{
p1 (0.0 0.0 0.0); // first point for the axis of rotation
p2 (0.0 0.0 1.0); // second point for the axis of rotation
omega 1.256; // rotation speed (rad/s) => 12 rpm
}
}
surfaces
{
cylinder
{
type cylinderWall;
p1 (0.0 0.0 0.0);
p2 (0.0 0.0 1.6);
radius1 0.2;
radius2 0.2;
resolution 24;
material wallMat;
motion rotAxis;
type cylinderWall; // type of the wall
p1 (0.0 0.0 0.0); // begin point of cylinder axis
p2 (0.0 0.0 1.6); // end point of cylinder axis
radius1 0.2; // radius at p1
radius2 0.2; // radius at p2
resolution 24; // number of divisions
material wallMat; // material name of this wall
motion rotAxis; // motion component name
}
/*
This is a plane wall at the rear end of cylinder
*/
wall1
{
type planeWall;
p1 (-0.2 -0.2 0.0);
p2 ( 0.2 -0.2 0.0);
p3 ( 0.2 0.2 0.0);
p4 (-0.2 0.2 0.0);
material wallMat;
motion rotAxis;
type planeWall; // type of the wall
p1 (-0.2 -0.2 0.0); // first point of the wall
p2 ( 0.2 -0.2 0.0); // second point
p3 ( 0.2 0.2 0.0); // third point
p4 (-0.2 0.2 0.0); // fourth point
material wallMat; // material name of the wall
motion rotAxis; // motion component name
}
/*
This is a plane wall at the front end of cylinder
*/
wall2
{
type planeWall;
p1 (-0.2 -0.2 1.6);
p2 ( 0.2 -0.2 1.6);
p3 ( 0.2 0.2 1.6);
p4 (-0.2 0.2 1.6);
material wallMat;
motion rotAxis;
}
type planeWall; // type of the wall
}
p1 (-0.2 -0.2 1.6); // first point of the wall
// information for rotatingAxisMotion motion model
rotatingAxisMotionInfo
{
rotAxis
{
p1 (0.0 0.0 0.0);
p2 (0.0 0.0 1.0);
omega 1.256; // rotation speed (rad/s) => 12 rpm
p2 ( 0.2 -0.2 1.6); // second point
p3 ( 0.2 0.2 1.6); // third point
p4 (-0.2 0.2 1.6); // fourth point
material wallMat; // material name of the wall
motion rotAxis; // motion component name
}
}

View File

@ -5,10 +5,10 @@
objectName particlesDict;
objectType dictionary;
fileFormat ASCII;
setFields
{
defaultValue
{
velocity realx3 (0 0 0); // linear velocity (m/s)
@ -23,22 +23,25 @@ setFields
positionParticles
{
method positionOrdered;
method ordered;
maxNumberOfParticles 4000001;
mortonSorting Yes;
cylinder // box for positioning particles
{
p1 ( 0.0 0.0 0.01); // lower corner point of the box
p2 ( 0.0 0.0 1.59); // upper corner point of the box
radius 0.195;
}
positionOrderedInfo
orderedInfo
{
diameter 0.003; // minimum space between centers of particles
numPoints 4000000; // number of particles in the simulation
axisOrder (z x y); // axis order for filling the space with particles
}
regionType cylinder; // other options: box and sphere
cylinderInfo // cylinder for positioning particles
{
p1 (0.0 0.0 0.01); // lower corner point of the box
p2 (0.0 0.0 1.59); // upper corner point of the box
radius 0.195; // radius of cylinder
}
}

View File

@ -3,9 +3,10 @@
| copyright: www.cemf.ir |
\* ------------------------------------------------------------------------- */
objectName settingsDict;
objectType dictionary;;
run rotatingDrum_1;
objectType dictionary;
fileFormat ASCII;
/*---------------------------------------------------------------------------*/
run rotatingDrum_4MParticles;
dt 0.00001; // time step for integration (s)
@ -19,14 +20,15 @@ timePrecision 5; // maximum number of digits for time folder
g (0 -9.8 0); // gravity vector (m/s2)
domain
{
min (-0.2 -0.2 -0.0);
max ( 0.2 0.2 1.6);
}
includeObjects (diameter); // save necessary (i.e., required) data on disk
integrationMethod AdamsBashforth3; // integration method
// exclude unnecessary data from saving on disk
excludeObjects (rVelocity.dy1 pStructPosition.dy1 pStructVelocity.dy1);
integrationMethod AdamsBashforth2; // integration method
writeFormat binary; // data writting format (ascii or binary)
timersReport Yes;
timersReportInterval 0.01;
timersReportInterval 0.05;

View File

@ -1,5 +1,5 @@
# Problem Definition
The problem is to simulate a rotating drum with the diameter **0.24 m**, the length **0.1 m** and **6** Baffles, rotating at **15 rpm**. This drum is filled with **20000** Particles.The timestep for integration is **0.00001 s**. There are 2 types of Particles in this drum each are being inserted during simulation to fill the drum.
# Problem Definition (v-1.0)
The problem is to simulate a rotating drum with a diameter of 0.24 m, a length of 0.1 m and 6 baffles rotating at 15 rpm. This drum is filled with 20000 particles, the integration time step is 0.00001 s. There are 2 types of particles in this drum, each of which is inserted during the simulation to fill the drum.
* **12500** Particles with **4 mm** diameter, at the rate of 12500 particles/s for 1 sec.
* **7500** Particles with **5mm** diameter, at the rate of 7500 particles/s for 1 sec.
@ -15,10 +15,10 @@ The problem is to simulate a rotating drum with the diameter **0.24 m**, the len
</html>
# Setting up the Case
As it has been explained in the previous cases, the simulation case setup is based on text-based scripts. Here, the simulation case setup are sotred in three folders: `caseSetup`, `setting` and `stl` (see the above folders).
As it has been explained in the previous cases, the simulation case setup is based on text-based scripts. Here, the simulation case setup are sorted in three folders: `caseSetup`, `setting` and `stl`.
## Defining small and large particles
Then in the `caseSetup/sphereShape` the diameter and the material name of the particles are defined. Two sizes are defined: 4 and 5 mm.
Then in the `caseSetup/shapes` the diameter and the material name of the particles are defined. Two sizes are defined: 4 and 5 mm.
```C++
// names of shapes
names (smallSphere largeSphere);
@ -30,7 +30,7 @@ materials (lightMat heavyMat);
## Particle Insertion
In this case we have two region for inserting our particles. In the both region we define rate of insertion, start and end time of insertion, information for the volume of the space throught which particles are being inserted. The insertion phase in the simulation is performed between times 0 and 1 seconds.
In this case we have two regions for inserting the particles. In both regions we define the insertion rate, the start and end time of the insertion, information about the volume of space through which the particles are inserted. The insertion phase in the simulation is performed between times 0 and 1 second.
For example, for the insertion region for inserting light particles is shown below.
<div align="center">
@ -43,7 +43,8 @@ in <b>caseSetup/particleInsertion</b> file
layerrightregion
{
// type of insertion region
type cylinderRegion;
timeControl simulationTime;
regionType cylinder;
// insertion rate (particles/s)
rate 12500;
// Start time of LightParticles insertion (s)
@ -51,9 +52,9 @@ layerrightregion
// End time of LightParticles insertion (s)
endTime 1;
// Time Interval of LightParticles insertion (s)
interval 0.025;
insertionInterval 0.025;
cylinderRegionInfo
cylinderInfo
{
// Coordinates of cylinderRegion (m,m,m)
p2 (-0.15 0.25 0.05);
@ -64,7 +65,7 @@ layerrightregion
}
```
## Interaction between particles and walls
In `caseSetup/interaction` file, material names and properties and interaction parameters are defined: interaction between the particles and the shell of rotating drum. Since we are defining 3 materials for simulation, the interaction matrix is 3x3, while we are only required to enter upper-triangle elements (interactions are symetric).
The `caseSetup/interaction` file defines the material names and properties as well as the interaction parameters: the interaction between the particles and the shell of the rotating drum. Since we define 3 materials for simulation, the interaction matrix is 3x3, while we only need to enter upper triangle elements (interactions are symmetric).
```C++
// a list of materials names
@ -93,10 +94,6 @@ densities (1000 1500 2500);
en (0.97 0.97 0.85
0.97 0.85
1.00);
// coefficient of tangential restitution
et (1.0 1.0 1.0
1.0 1.0
1.0);
// dynamic friction
mu (0.65 0.65 0.35
0.65 0.35
@ -166,7 +163,8 @@ surfaces
In this part of `geometryDict` the information of rotating axis and speed of rotation are defined. The start of rotation is at 2 s. The first 2 seconds of simulation is for allowing particles to settle donw in the drum.
```C++
rotatingAxisMotionInfo
motionModel rotatingAxis;
rotatingAxisInfo
{
rotAxis
{
@ -184,9 +182,9 @@ rotatingAxisMotionInfo
}
```
## Performing Simulation
To perform simulations, enter the following commands one after another in the terminal.
To run simulations, type the following commands in the terminal one at a time.
Enter `$ particlesPhasicFlow` command to create the initial fields for particles.
Enter `$ geometryPhasicFlow` command to create the Geometry.
At last, enter `$ sphereGranFlow` command to start the simulation.
After finishing the simulation, you can use `$ pFlowtoVTK` to convert the results into vtk format storred in ./VTK folder.
Enter `particlesPhasicFlow` command to create the initial fields for particles.
Enter `geometryPhasicFlow` command to create the Geometry.
At last, enter `sphereGranFlow` command to start the simulation.
After finishing the simulation, you can use `pFlowtoVTK` to convert the results into vtk format stored in ./VTK folder.

View File

@ -53,10 +53,6 @@ model
0.97 0.85
1.00); // coefficient of normal restitution
et (1.0 1.0 1.0
1.0 1.0
1.0); // coefficient of tangential restitution
mu (0.65 0.65 0.35
0.65 0.35
0.35); // dynamic friction

View File

@ -8,8 +8,6 @@ fileFormat ASCII;
/*---------------------------------------------------------------------------*/
active Yes; // is insertion active -> Yes or No
checkForCollision No; // is checked -> Yes or No
/*
Two layers of particles are packed one-by-one using 1 insertion steps
*/
@ -31,8 +29,7 @@ layerrightregion // Right Layer Region
cylinderInfo
{
p2 (-0.15 0.25 0.05); //
p2 (-0.15 0.25 0.05); // Top of cylinderRegion (m,m,m)
p1 (-0.15 0.24 0.05); // Bottom of cylinderRegion (m,m,m)

View File

@ -13,25 +13,8 @@ globalBox // Simulation domain: every par
max (-0.068 0.355 0.125); // upper corner point of the box
}
decomposition
{
direction z;
}
boundaries
{
neighborListUpdateInterval 50; /* Determines how often (how many iterations) do you want to
rebuild the list of particles in the neighbor list
of all boundaries in the simulation domain */
updateInterval 10; // Determines how often do you want to update the new changes in the boundary
neighborLength 0.004; // The distance from the boundary plane within which particles are marked to be in the boundary list
left
{
type exit; // other options: periodic, reflective

View File

@ -79,4 +79,3 @@ surfaces
motion rotAxis; // motion component name
}
}

View File

@ -52,27 +52,7 @@ setFields
positionParticles // positions particles
{
method ordered; // other options: random and empty
mortonSorting Yes; // perform initial sorting based on morton code?
orderedInfo
{
diameter 0.005; // minimum space between centers of particles
numPoints 20000; // number of particles in the simulation
axisOrder (z y x); // axis order for filling the space with particles
}
regionType box; // other options: cylinder and sphere
boxInfo // box information for positioning particles
{
min (-0.08 -0.08 0.015); // lower corner point of the box
max ( 0.08 0.08 0.2); // upper corner point of the box
}
method empty; // other options: random and ordered
}

View File

@ -20,7 +20,8 @@ timePrecision 6; // maximum number of digits for time
g (0 -9.8 0); // gravity vector (m/s2)
includeObjects (diameter); // save necessary (i.e., required) data on disk
// save necessary (i.e., required) data on disk
includeObjects (diameter);
// exclude unnecessary data from saving on disk
excludeObjects (rVelocity.dy1 pStructPosition.dy1 pStructVelocity.dy1);
@ -31,4 +32,4 @@ writeFormat ascii; // data writting format (ascii or binary
timersReport Yes; // report timers (Yes or No)
timersReportInterval 0.01; // time interval for reporting timers
timersReportInterval 0.1; // time interval for reporting timers

View File

@ -20,11 +20,6 @@ timePrecision 6; // maximum number of digits for
g (0 -9.8 0); // gravity vector (m/s2)
/*
Simulation domain
every particles that goes outside this domain is deleted.
*/
includeObjects (diameter); // save necessary (i.e., required) data on disk
// exclude unnecessary data from saving on disk