<!-------- @HEADER
!
! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!
! Zoltan Toolkit for Load-balancing, Partitioning, Ordering and Coloring
! Copyright 2012 Sandia Corporation
!
! Under the terms of Contract DE-AC04-94AL85000 with Sandia Corporation,
! the U.S. Government retains certain rights in this software.
!
! Redistribution and use in source and binary forms, with or without
! modification, are permitted provided that the following conditions are
! met:
!
! 1. Redistributions of source code must retain the above copyright
! notice, this list of conditions and the following disclaimer.
!
! 2. Redistributions in binary form must reproduce the above copyright
! notice, this list of conditions and the following disclaimer in the
! documentation and/or other materials provided with the distribution.
!
! 3. Neither the name of the Corporation nor the names of the
! contributors may be used to endorse or promote products derived from
! this software without specific prior written permission.
!
! THIS SOFTWARE IS PROVIDED BY SANDIA CORPORATION "AS IS" AND ANY
! EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
! IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
! PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL SANDIA CORPORATION OR THE
! CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
! EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
! PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
! PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
! LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
! NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
! SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
!
! Questions? Contact Karen Devine kddevin@sandia.gov
! Erik Boman egboman@sandia.gov
!
! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!
! @HEADER
------->
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Mozilla/4.7 [en] (X11; U; SunOS 5.6 sun4m) [Netscape]">
<meta name="sandia.approval_type" content="formal">
<meta name="sandia.approved" content="SAND2007-4748W">
<meta name="author" content="Zoltan PI">
<title>Zoltan User's Guide: Library Usage</title>
</head>
<body bgcolor="#FFFFFF">
<div align=right><b><i><a href="ug.html">Zoltan User's Guide</a>
| <a href="ug_cpp.html">Next</a> | <a href="ug_intro.html">Previous</a></i></b></div>
<h2>
<a NAME="Using the Library"></a>Using the Zoltan library</h2>
This section contains information needed to use the Zoltan library with
applications:
<blockquote>
<a href="#System Requirements">System requirements</a>
<br><a href="#Building the Library">Building the Zoltan library</a>
<br><a href="#Testing the Library">Testing the Zoltan library</a>
<br><a href="#ReportingBugs">Reporting bugs</a> in Zoltan
<br><a href="#Incorporating Zoltan">Incorporating Zoltan into Applications</a>
<br><a href="#Building Applications">Building applications</a>
that use Zoltan
<br><a href="#Data Types for Object IDs">Data types</a> for global and
local IDs
<br><a href="ug_cpp.html">C++ interface</a>
<br><a href="ug_fortran.html">F90 interface</a>
</blockquote>
<hr>
<h3>
<a NAME="System Requirements"></a>System Requirements</h3>
Zoltan was designed to run on parallel computers and clusters of workstations.
The most common builds and installations of Zoltan use the following:
<ul>
<li>
ANSI C or C++ compiler.</li>
<li>
<a href="https://www-unix.mcs.anl.gov/mpi/">MPI</a> library for message
passing (version 1.1 or higher), such as MPICH, OpenMPI or LAM.</li>
<li>
A Unix-like operating system (e.g., Linux or Mac OS X) and
<i>gmake</i> (GNU Make) are recommended to build the library.</li>
<li>
A Fortran90 compatible compiler is required if you wish to
<a href="ug_fortran.html">use Zoltan
with Fortran applications</a>.</li>
</ul>
Zoltan has been tested on a variety of platforms, including Linux, Mac OS X,
a variety of clusters, and Sandia's ASC RedStorm</a> machine.
Builds for Windows platforms are available as part of the
<a href=#CMake>Trilinos CMake build system</a>.
<hr>
<h3>
<a NAME="Building the Library"></a>Building the Zoltan Library</h3>
The Zoltan library is implemented in ANSI C and can be compiled with any
ANSI C compiler.
In Zoltan, there are two build environments currently supported:
an <a href="#Autotools">Autotools build environment</a>
and a <a href=#CMake">CMake build environment</a>
used with the <a href="https://trilinos.sandia.gov">Trilinos</a>
framework.
The Autotools build environment can be used to build Zoltan in a stand-alone
installation; the CMake build environment must be used within Trilinos.
<p>
<h4><a NAME="Autotools"></a>Using Autotools to Build Zoltan</h4>
Users should not run autotools directly in the main Zoltan directory;
rather they should create a build-directory (e.g., a subdirectory of
the main Zoltan directory) in which they configure and build Zoltan.
Say, for example, a user creates a directory called BUILD_DIR in the
Zoltan directory. Then, to configure and build zoltan, the user would
<blockquote>
cd zoltan/BUILD_DIR <br>
../configure {options described below} <br>
make everything <br>
make install <br>
</blockquote>
Options to the configure command allow paths
to third-party libraries such as ParMETIS, PT-Scotch and PaToH to be specified.
Building with MPI compilers (e.g., mpicc) is the default for Autotools builds
of Zoltan; many options allow specification of particular MPI paths and
compilers.
<p>
Users desiring a <a href="ug_fortran.html">Fortran90 interface</a>
to Zoltan must
specify the "--enable-f90interface" option.
<p>
All options can be seen
with the following command issued in the zoltan/BUILD_DIR directory:
<blockquote>
../configure --help
</blockquote>
<p>
The following script is an example of configuration and build commands
using Autotools. It specifies that Zoltan should be built with both
the <a href="ug_alg_parmetis.html">ParMETIS</a> and
<a href="ug_alg_ptscotch.html">PT-Scotch</a> interfaces.
Paths to both ParMETIS and PT-Scotch are given.
The prefix option states where Zoltan should be installed;
in this example, Zoltan's include files will be installed in
/homes/username/zoltan/BUILD_DIR/include, and the libraries
will be installed in /homes/username/zoltan/BUILD_DIR/lib.
This examples assumes the script is run from
/homes/username/zoltan/BUILD_DIR.
<blockquote>
#<br>
../configure \<br>
--prefix=/homes/username/zoltan/BUILD_DIR \<br>
--with-gnumake \<br>
--with-scotch \<br>
--with-scotch-incdir="/Net/local/proj/zoltan/arch/all/src/Scotch5" \<br>
--with-scotch-libdir="/Net/local/proj/zoltan/arch/linux64/lib/openmpi/Scotch5" \<br>
--with-parmetis \<br>
--with-parmetis-incdir="/Net/local/proj/zoltan/arch/all/src/ParMETIS3" \<br>
--with-parmetis-libdir="/Net/local/proj/zoltan/arch/linux64/lib/openmpi/ParMETIS3" <br>
make everything <br>
make install
</blockquote>
<p>
The configure script also allows you to specify the data type for a
global identifier. The choices are
unsigned int, unsigned long, and unsigned long long.
The default data type is unsigned int. If your
space of global identifiers requires more than 32 bits, you can specify a 64-bit data type.
<p>
<blockquote>
--with-id-type=uint<br>
--with-id-type=ulong<br>
--with-id-type=ullong<br>
</blockquote>
<p>
Support of 64-bit identifiers is new as of Zoltan version 3.5.
At this point in time all methods except for
<a href="ug_alg_reftree.html">refinement tree partitioning</a>
have been modified to work with 64-bit identifiers. Zoltan's
Fortran90 interface does not yet support 64-bit identifiers.
<p>
More examples are in the directory zoltan/SampleConfigurationScripts.
<p>
After the configuration is done in
the build directory, object files and executables can be removed with
<i>make clean</i>; the same configuration will be used for subsequent builds.
Configuration information is removed with <i>make distclean</i>.
<p>
<h4><a NAME="CMake"></a>Using CMake to Build Zoltan</h4>
Zoltan can be built as part of the Trilinos framework using the
CMake build system. CMake builds will succeed only when Zoltan is
in the Trilinos directory structure (as when downloaded with Trilinos).
Users should not run CMake directly in the main Zoltan directory;
rather they should create a build-directory (e.g., a subdirectory of
the main Trilinos directory) in which they configure and build Zoltan.
Say, for example, a user creates a directory called BUILD_DIR in the
Trilinos directory. Then, to configure and build zoltan, the user would
<blockquote>
cd Trilinos/BUILD_DIR <br>
cmake \ <br>
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \ <br>
-D Trilinos_ENABLE_Zoltan:BOOL=ON \ <br>
{options described below} \ <br>
.. <br>
make <br>
make install <br>
</blockquote>
<p>
CMake also allows you to specify the data type for a
global identifier. The choices are
unsigned int, unsigned long, and unsigned long long.
The default data type for a global identifier is unsigned int. The options to set the global identifier
data type are shown below.
<p>
<blockquote>
-D Zoltan_ENABLE_UINT_IDS:Bool=ON<br>
-D Zoltan_ENABLE_ULONG_IDS:Bool=ON<br>
-D Zoltan_ENABLE_ULLONG_IDS:Bool=ON<br>
</blockquote>
<p>
Support of 64-bit identifiers is new as of Zoltan version 3.5.
At this point in time all methods except for
<a href="ug_alg_reftree.html">refinement tree partitioning</a>
have been modified to work with 64-bit identifiers. Zoltan's Fortran90
interface does not yet support 64-bit identifiers.
<p>
<b>Serial</b> builds are the default in Trilinos; for serial builds, Zoltan
builds and links with the siMPI library provided by Pat Miller in the Zoltan
distribution. More commonly, Zoltan users desire <b>parallel</b> builds with
MPI libraries such as OpenMPI or MPICH. For such builds, users must specify
CMake option
<blockquote>
-D TPL_ENABLE_MPI:BOOL=ON
</blockquote>
Trilinos also defaults to using a Fortran compiler, but Fortran is not
required to build Zoltan; the option to disable this check is<br>
-D Trilinos_ENABLE_Fortran:BOOL=OFF
<p>
Other options to the cmake command allow paths
to third-party libraries such as ParMETIS, PT-Scotch and PaToH to be specified.
<p>
Users desiring a <a href="ug_fortran.html">Fortran90 interface</a>
to Zoltan must
specify the option<br>
-D Zoltan_ENABLE_F90INTERFACE:BOOL=ON<br>
<p>
All options can be seen
with the following command issued in the Trilinos/BUILD_DIR directory:
<blockquote>
rm CMakeCache.txt<br>
cmake -LAH -D Trilinos_ENABLE_Zoltan:BOOL=ON ..
</blockquote>
<p>
The following script is an example of configuration and build commands
using CMake. It specifies that Zoltan should be built with both
the <a href="ug_alg_parmetis.html">ParMETIS</a> and
<a href="ug_alg_ptscotch.html">PT-Scotch</a> interfaces.
Paths to both ParMETIS and PT-Scotch are given.
The prefix option states where Zoltan should be installed;
in this example, Zoltan's include files will be installed in
/homes/username/Trilinos/BUILD_DIR/include, and the libraries
will be installed in /homes/username/Trilinos/BUILD_DIR/lib.
This examples assumes the script is run from
/homes/username/Trilinos/BUILD_DIR.
<blockquote>
#<br>
cmake \ <br>
-D CMAKE_INSTALL_PREFIX:FILEPATH="/home/username/Trilinos/BUILD_DIR" \ <br>
-D TPL_ENABLE_MPI:BOOL=ON \ <br>
-D CMAKE_C_FLAGS:STRING="-m64 -g" \ <br>
-D CMAKE_CXX_FLAGS:STRING="-m64 -g" \ <br>
-D CMAKE_Fortran_FLAGS:STRING="-m64 -g" \ <br>
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \ <br>
-D Trilinos_ENABLE_Zoltan:BOOL=ON \ <br>
-D Zoltan_ENABLE_EXAMPLES:BOOL=ON \ <br>
-D Zoltan_ENABLE_TESTS:BOOL=ON \ <br>
-D Zoltan_ENABLE_ParMETIS:BOOL=ON \ <br>
-D ParMETIS_INCLUDE_DIRS:FILEPATH="/home/username/code/ParMETIS3_1" \ <br>
-D ParMETIS_LIBRARY_DIRS:FILEPATH="/home/username/code/ParMETIS3_1" \ <br>
-D Zoltan_ENABLE_Scotch:BOOL=ON \ <br>
-D Scotch_INCLUDE_DIRS:FILEPATH="/home/username/code/scotch_5.1/include" \ <br>
-D Scotch_LIBRARY_DIRS:FILEPATH="/home/username/code/scotch_5.1/lib" \ <br>
.. <br>
make <br>
make install
</blockquote>
<p>
More examples are in the directory zoltan/SampleCmakeScripts.
More details of CMake use in Trilinos are
in <br>
Trilinos/cmake/TrilinosCMakeQuickstart.txt.
<hr>
<h3>
<a NAME="Testing the Library"></a>Testing the Zoltan Library</h3>
The <I>examples</I> directory contains simple C and C++ examples which use
the Zoltan library. The makefile in this directory has three targets:
These examples are built automatically when the
<a href="#Autotools">Autotools build environment</a> or
<a href="#CMake">CMake build environment</a> is used.
<p>
The "right" answer for these tests depends on the number of processes with
which you run the tests. In general, if they compile successfully,
run quickly (in seconds), and produce reasonable looking output, then
Zoltan is built successfully.
<hr>
<h3>
<a NAME="ReportingBugs"></a>Reporting Bugs in Zoltan</h3>
Zoltan uses <a href="https://www.bugzilla.org">Bugzilla</a> to collect
bug reports. Please read the <a href="../Zoltan_bugreport.html">instructions for reporting bugs</a> through the Zoltan Bugzilla database.
<p>
<hr>
<h3>
<a NAME="Incorporating Zoltan"></a>Incorporating Zoltan into Applications</h3>
Incorporating Zoltan into applications requires three basic steps:
<ul>
<li>
Writing <a href="ug_query.html">query functions</a>
that return information about the application to Zoltan.
<li>
<a href="ug_interface_init.html">Initializing</a> Zoltan, <a href="ug_interface_init.html#Zoltan_Create">creating a
Zoltan object</a>, and
<a href="ug_interface_init.html#Zoltan_Set_Param">setting
appropriate parameters</a>.
<li>
Calling Zoltan tools to perform <a href="ug_interface_lb.html">partitioning</a>, <a href="ug_interface_order.html">ordering</a>, <a href="ug_interface_mig.html">migration</a>, <a href="ug_interface_color.html">coloring</a>, etc.
</ul>
The set of <a href="ug_query.html">query functions</a>
needed by an application depends on the
particular tools (e.g., <a href="ug_interface_lb.html">partitioning</a>,
<a href="ug_interface_order.html">ordering</a>) used and on
the <a href="ug_alg.html">algorithms</a>
selected within the tools. Not all query functions are needed by
every application. See documentation on tools and algorithms to determine
which query functions are needed.
<hr>
<h3>
<a NAME="Building Applications"></a>Building Applications that use Zoltan</h3>
The C library interface is described in the include file <i>include/zoltan.h</i>;
this file should be included in all C application source files that call
Zoltan library routines.
<p>
The <a href="ug_cpp.html">C++ interface</a> to
Zoltan is implemented in header files which define classes that
wrap the Zoltan C library. The file <I>include/zoltan_cpp.h</I> defines the
<B>Zoltan</B> class which encapsulates a load balancing data structure and the
Zoltan load balancing functions which operate upon it. Include this header file
instead in your C++ application. Note that C++ applications should call the
C function <B><a href="ug_interface_init.html#Zoltan_Initialize">Zoltan_Initialize</a></B> before creating a <B>Zoltan</B> object.
<p>
<a href="ug_fortran_apps.html">Fortran applications</a> must USE
<a href="ug_fortran_api.html#fortran ug api zoltan module">module zoltan</a> and
specify the Zoltan installation's <i>include</i> directory
as a directory to be searched for module information files.
<p>
The C, C++ or Fortran application should then be linked with the Zoltan library
(built with Fortran support in the Fortran case) by including
<blockquote><i>-lzoltan </i></blockquote>
in the linking command for the application.
Communication within
Zoltan is performed through MPI, so appropriate MPI libraries must be linked
with the application. Third-party libraries, such as <a href="ug_alg_parmetis.html">ParMETIS</a>, <a href="ug_alg_ptscotch.html">PT-Scotch</a>
and <a href="ug_alg_patoh.html">PaToH</a>, must be also be
linked with the application if they were included in compilation of the
Zoltan library.
<p>
The installed files <i>include/Makefile.export.zoltan*</i> contain macros that
can specify Zoltan paths and libraries in an application's Makefiles.
Using these files, applications can be assured they are using the same
build options that were used when Zoltan was built.
<hr>
<h3>
<a NAME="Data Types for Object IDs"></a>Data Types for Object IDs</h3>
Application query functions and application callable library functions
use global and local identifiers (IDs) for objects. <i>All objects to be
used in load balancing must have unique global IDs.</i> Zoltan stores an
ID as an array of ZOLTAN_ID_TYPE. The default for ZOLTAN_ID_TYPE is unsigned
int, but configuration parameters can select unsigned long or unsigned long long
as the type; see above for the appropriate configuration flags for
<a href="#Autotools">Autotools</a> and <a href="#CMake">CMake</a>.
The number of entries in these arrays
can be set using the <a href="ug_param.html#NUM_GID_ENTRIES">NUM_GID_ENTRIES</a>
and <a href="ug_param.html#NUM_LID_ENTRIES">NUM_LID_ENTRIES</a> parameters;
by default, one ZOLTAN_ID_TYPE represents an ID. Applications may use
whatever format is most convenient to store their IDs; the IDs can then
be converted to and from Zoltan's ID format in the <a href="ug_query.html">application-registered
query functions</a>.
<p>
Definitions of ZOLTAN_ID_TYPE and ZOLTAN_ID_MPI_TYPE
are in <i>include/zoltan_types.h</i>; they can
be used by an application for memory allocation, MPI communication, and
as arguments to
<a href="ug_interface.html">load-balancing interface functions</a>
and
<a href="ug_query.html">application-registered query functions</a>.
In the Fortran interface, IDs are passed as arrays of integers since unsigned
integers are not supported in Fortran. See the description of the <a href="ug_fortran_api.html#fortran ug api IDs">Fortran
interface</a> for more details.
<p>The local IDs passed to Zoltan are not used by the library; they are
provided for the convenience of the application and can contain any information
desired by the application. For instance, local array indices for objects
may be passed as local IDs, enabling direct access to object data in the
query function routines. See the <a href="ug_query.html">application-registered
query functions</a> for more details. The source code distribution contains
an example application <i><a href="../dev_html/dev_driver.html">zdrive</a></i>
in which global IDs are integers and local IDs are local array indices.
One may choose not to use local ids at all, in which case <a href="ug_param.html#NUM_LID_ENTRIES">NUM_LID_ENTRIES</a>
may be set to zero.
<p>Some Zoltan routines (e.g.,
<a href="ug_interface_lb.html#Zoltan_LB_Partition"><b>Zoltan_LB_Partition</b></a>
and
<a href="ug_interface_mig.html#Zoltan_Invert_Lists"><b>Zoltan_Invert_Lists</b></a>)
allocate arrays of type <b>ZOLTAN_ID_PTR</b> and return them to the application.
Others (e.g., <b><a href="ug_interface_order.html#Zoltan_Order">Zoltan_Order</a></b> and
<a href="ug_util_dd.html#DD_Find"><b>Zoltan_DD_Find</b></a>) require
the application to allocate memory for IDs. Memory for IDs can be allocated
as follows:
<blockquote><tt>
ZOLTAN_ID_PTR gids;<br>
int num_gids, int num_gid_entries;<br>
gids = (ZOLTAN_ID_PTR) <a href="ug_util_mem.html#Zoltan_Malloc">ZOLTAN_MALLOC</a>(num_gids * num_gid_entries * sizeof(ZOLTAN_ID_TYPE);<br>
</tt></blockquote>
The system call <i>malloc</i> may be used instead of
<a href="ug_util_mem.html#Zoltan_Malloc"><b>ZOLTAN_MALLOC</b></a>.
<hr WIDTH="100%">[<a href="ug.html">Table of Contents</a> | <a href="ug_cpp.html">Next:
C++ Interface</a> | <a href="ug_intro.html">Previous:
Introduction</a> | <a href="https://www.sandia.gov/general/privacy-security/index.html">Privacy and Security</a>]
</body>
</html>