Step 6: Error 8#
Error Message#
1$ pylith step06_twofaults.cfg
2
3 >> software/pylith-debug/lib/python3.12/site-packages/pylith/apps/PyLithApp.py:76:main
4 -- info (application-flow)
5 -- Running on 1 process(es).
6 >> src/cig/pylith/libsrc/pylith/utils/PetscOptions.cc:251:static void pylith::utils::_PetscOptions::write(pythia::journal::info_t&, const char*, const pylith::utils::PetscOptions&)
7 -- info (application-flow)
8 -- Setting PETSc options:
9 dm_reorder_section = true
10 dm_reorder_section_type = cohesive
11 ksp_atol = 1.0e-7
12 ksp_converged_reason = true
13 ksp_error_if_not_converged = true
14 ksp_gmres_restart = 100
15 ksp_guess_pod_size = 8
16 ksp_guess_type = pod
17 ksp_rtol = 1.0e-14
18 mg_fine_ksp_max_it = 5
19 mg_fine_pc_type = vpbjacobi
20 pc_type = gamg
21 snes_atol = 5.0e-7
22 snes_converged_reason = true
23 snes_error_if_not_converged = true
24 snes_monitor = true
25 snes_rtol = 1.0e-14
26 ts_error_if_step_fails = true
27 ts_exact_final_time = matchstep
28 ts_monitor = true
29 ts_type = beuler
30 viewer_hdf5_collective = true
31
32 >> src/cig/pylith/libsrc/pylith/meshio/MeshIOPetsc.cc:204:virtual void pylith::meshio::MeshIOPetsc::_read()
33 -- info (application-flow)
34 -- Component 'meshiopetsc.reader': Reading finite-element mesh from 'mesh_tri.msh'.
35 >> src/cig/pylith/libsrc/pylith/meshio/MeshIO.cc:76:void pylith::meshio::MeshIO::read(pylith::topology::Mesh*, bool)
36 -- info (application-flow)
37 -- Component 'meshiopetsc.reader': Domain bounding box:
38 (-100000, 100000)
39 (-100000, 0)
40 >> src/cig/pylith/libsrc/pylith/problems/TimeDependent.cc:316:virtual void pylith::problems::TimeDependent::verifyConfiguration() const
41 -- info (application-flow)
42 -- Component 'timedependent.problem': Verifying problem configuration.
43 >> software/pylith-debug/lib/python3.12/site-packages/pylith/problems/Problem.py:238:_printInfo
44 -- info (application-flow)
45 -- Scales for nondimensionalization:
46 Length scale: 2500*m
47 Displacement scale: 1*m
48 Time scale: 3.15576e+09*s
49 Rigidity scale: 1e+10*m**-1*kg*s**-2
50 Temperature scale: 1*K
51 >> src/cig/pylith/libsrc/pylith/problems/TimeDependent.cc:342:virtual void pylith::problems::TimeDependent::initialize()
52 -- info (application-flow)
53 -- Component 'timedependent.problem': Initializing problem.
54[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
55[0]PETSC ERROR: Error in external library
56[0]PETSC ERROR: Could not find values for initiation_time at ( -21860.3 -27621) in spatial database 'Fault rupture for main fault'.
57[0]PETSC ERROR: WARNING! There are unused option(s) set! Could be the program crashed before usage or a spelling mistake, etc!
58[0]PETSC ERROR: Option left: name:-ksp_atol value: 1.0e-7 source: code
59[0]PETSC ERROR: Option left: name:-ksp_converged_reason (no value) source: code
60[0]PETSC ERROR: Option left: name:-ksp_error_if_not_converged (no value) source: code
61[0]PETSC ERROR: Option left: name:-ksp_gmres_restart value: 100 source: code
62[0]PETSC ERROR: Option left: name:-ksp_guess_pod_size value: 8 source: code
63[0]PETSC ERROR: Option left: name:-ksp_guess_type value: pod source: code
64[0]PETSC ERROR: Option left: name:-ksp_rtol value: 1.0e-14 source: code
65[0]PETSC ERROR: Option left: name:-mg_fine_ksp_max_it value: 5 source: code
66[0]PETSC ERROR: Option left: name:-mg_fine_pc_type value: vpbjacobi source: code
67[0]PETSC ERROR: Option left: name:-pc_type value: gamg source: code
68[0]PETSC ERROR: Option left: name:-snes_atol value: 5.0e-7 source: code
69[0]PETSC ERROR: Option left: name:-snes_converged_reason (no value) source: code
70[0]PETSC ERROR: Option left: name:-snes_error_if_not_converged (no value) source: code
71[0]PETSC ERROR: Option left: name:-snes_monitor (no value) source: code
72[0]PETSC ERROR: Option left: name:-snes_rtol value: 1.0e-14 source: code
73[0]PETSC ERROR: Option left: name:-ts_error_if_step_fails (no value) source: code
74[0]PETSC ERROR: Option left: name:-ts_exact_final_time value: matchstep source: code
75[0]PETSC ERROR: Option left: name:-ts_monitor (no value) source: code
76[0]PETSC ERROR: Option left: name:-ts_type value: beuler source: code
77[0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting.
78[0]PETSC ERROR: PETSc Development Git Revision: v3.25.1-168-g70613f6caab Git Date: 2026-05-21 15:15:28 +0000
79[0]PETSC ERROR: software/pylith-debug/bin/mpinemesis with 1 MPI process(es) and PETSC_ARCH arch-pylith-debug on igskci164warlng.gs.doi.net by baagaard Tue May 26 12:55:27 2026
80[0]PETSC ERROR: Configure options: --PETSC_ARCH=arch-pylith-debug --with-debugging=1 --with-clanguage=c --with-mpi-compilers=1 --with-shared-libraries=1 --with-64-bit-points=1 --with-large-file-io=1 --with-lgrind=0 --download-parmetis=1 --download-metis=1 --download-triangle --download-ml=1 --download-superlu=1 --with-fc=0 --download-f2cblaslapack --with-hdf5=1 --with-hdf5-dir=software/pylith-debug --with-zlib=1 CFLAGS+=-g
81[0]PETSC ERROR: #1 static PetscErrorCode pylith::topology::FieldQuery::queryDBPointFn(PylithInt, PylithReal, const PylithReal*, PylithInt, PylithScalar*, void*)() at src/cig/pylith/libsrc/pylith/topology/FieldQuery.cc:320
82[0]PETSC ERROR: #2 DMProjectPoint_Func_Private() at /software/baagaard/petsc-dev/src/dm/impls/plex/plexproject.c:143
83[0]PETSC ERROR: #3 DMProjectPoint_Private() at /software/baagaard/petsc-dev/src/dm/impls/plex/plexproject.c:545
84[0]PETSC ERROR: #4 DMProjectLocal_Generic_Plex() at /software/baagaard/petsc-dev/src/dm/impls/plex/plexproject.c:1083
85[0]PETSC ERROR: #5 DMProjectFunctionLocal_Plex() at /software/baagaard/petsc-dev/src/dm/impls/plex/plexproject.c:1114
86[0]PETSC ERROR: #6 DMProjectFunctionLocal() at /software/baagaard/petsc-dev/src/dm/interface/dm.c:8361
87[0]PETSC ERROR: #7 void pylith::topology::FieldQuery::queryDB()() at src/cig/pylith/libsrc/pylith/topology/FieldQuery.cc:227
88Fatal error. Calling MPI_Abort() to abort PyLith application.
89Traceback (most recent call last):
90 File "software/pylith-debug/lib/python3.12/site-packages/pylith/apps/PetscApplication.py", line 55, in onComputeNodes
91 self.main(*args, **kwds)
92 File "software/pylith-debug/lib/python3.12/site-packages/pylith/apps/PyLithApp.py", line 85, in main
93 self.problem.initialize()
94 File "software/pylith-debug/lib/python3.12/site-packages/pylith/problems/Problem.py", line 212, in initialize
95 ModuleProblem.initialize(self)
96 File "software/pylith-debug/lib/python3.12/site-packages/pylith/problems/problems.py", line 165, in initialize
97 return _problems.Problem_initialize(self)
98 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
99RuntimeError: Error detected while in PETSc function.
100C++ traceback (13 frames):
101 [0] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith8topology10FieldQuery7queryDBEv+0x292) [0x75c8149a1782]
102 [1] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith10feassemble16AuxiliaryFactory15setValuesFromDBEv+0x2ca) [0x75c8148025e4]
103 [2] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith6faults6KinSrc10initializeERKNS_8topology5FieldERKNS_6scales6ScalesEPKN11spatialdata9geocoords8CoordSysE+0x51d) [0x75c8147880cd]
104 [3] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith6faults16FaultCohesiveKin20createAuxiliaryFieldERKNS_8topology5FieldERKNS2_4MeshE+0xa51) [0x75c81476f305]
105 [4] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith10feassemble10Integrator10initializeERKNS_8topology5FieldE+0x29f) [0x75c8147a7693]
106 [5] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith10feassemble19IntegratorInterface10initializeERKNS_8topology5FieldE+0x3d6) [0x75c8147ca87c]
107 [6] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith8problems7Problem10initializeEv+0x400) [0x75c81491d846]
108 [7] software/pylith-debug/lib/libpylith.so.0(_ZN6pylith8problems13TimeDependent10initializeEv+0x26f) [0x75c81492d011]
109 [8] software/pylith-debug/lib/python3.12/site-packages/pylith/problems/_problems.so(+0x15c30) [0x75c803d7ac30]
110 [9] software/pylith-debug/bin/mpinemesis(+0x15a51) [0x5717b3696a51]
111 [10] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x75c827029d90]
112 [11] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x75c827029e40]
113 [12] software/pylith-debug/bin/mpinemesis(+0x5805) [0x5717b3686805]
114
115--------------------------------------------------------------------------
116MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
117 Proc: [[4252,1],0]
118 Errorcode: -1
119
120NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
121You may or may not see output from other processes, depending on
122exactly when Open MPI kills them.
123--------------------------------------------------------------------------
124--------------------------------------------------------------------------
125prterun has exited due to process rank 0 with PID 0 on node igskci164warlng calling
126"abort". This may have caused other processes in the application to be
127terminated by signals sent by prterun (as reported here).
128--------------------------------------------------------------------------
129software/pylith-debug/bin/nemesis: mpiexec: exit 255
130software/pylith-debug/bin/pylith: software/pylith-debug/bin/nemesis: exit 1
Troubleshooting Strategy#
With the linear interpolation we get an error about not being able to find an initiation time for a point.
This suggests there are errors in our spatial database file related to interpolation.
We examine the header and data points for errors.
We notice that our points lie along a line (data dimension is 1), but our header has data-dim=2.
Resolution#
# Error
data-dim = 2
# Correct
data-dim = 1