Commit Graph
61 Commits
Author SHA1 Message Date
paboyle 9e6a4a4737 Assertion updates to macros (mostly) with backtrace.
WIlson flow to include options for DBW2, Iwasaki, Symanzik.
View logging for data assurance
2025-08-07 15:48:38 +00:00
paboyle 41f344bbd3 Merge with Christoph GPT checksum debug 2025-07-15 03:06:09 +00:00
paboyle bffb83c46e std::cout<<GridLogMessage<<"Debug:"<<std::endl;
std::cout<<GridLogMessage<<"  --dylib-map     : print dynamic library map, useful for interpreting signal backtraces "<<std::endl;
    std::cout<<GridLogMessage<<"  --heartbeat     : periodic itimer wakeup (interrupts stuck system calls!) "<<std::endl;
    std::cout<<GridLogMessage<<"  --signal-delay n : pause for n seconds after signal handling (useful to get ALL nodes in stuck state) "<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-stdout  : print stdout from EVERY node to file Grid.stdout/err.rank "<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-signals : catch sigsegv and print a blame report, handle SIGHUP with a backtrace to stderr"<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-heartbeat : periodically report backtrace "<<std::endl;

--dylib-map : Grid prints its dylib regions
--heartbeat : itimer based / SIGALRM wake up which seems to make Aurora
more stable
--debug-heartbeat : periodically report to stderr where we are in code

Now have libunwind option (configure: --with-unwind=<prefix>) to give an
Asynch-Signal safe backtrace. Avoid glibc backtrace due to mallocs.
2025-06-27 06:08:54 +00:00
paboyle b78fc73d19 Better signal handler 2025-06-27 06:07:34 +00:00
paboyle d418f78352 Making running on Aurora more debuggable 2025-05-23 20:58:16 +00:00
paboyle 1d22841811 Working on aurora, GPT issue turned up is fixed 2025-03-06 03:20:18 +00:00
paboyle ec1395a304 Better flight logging 2024-10-10 22:01:57 +00:00
paboyle 02c8178f16 Almost working on Aurora 2024-09-23 09:43:50 +00:00
paboyle 066544281f Deprecate UVM 2024-09-17 13:34:27 +00:00
Peter Boyle 5c3ace7c3e Merge branch 'develop' into feature/scidac-wp1 2024-04-30 05:26:06 -04:00
Peter Boyle 60b7f6c99d Flight recorder, resurrecting the "world famous" Britney test 2024-03-22 15:32:26 +00:00
Peter Boyle fab1efb48c More britney logging improvements 2024-03-19 14:36:21 +00:00
Peter Boyle 62e7bf024a Updated flight logging for Britney test 2024-03-12 20:10:04 +00:00
paboyle 2704b82084 Merge branch 'develop' of https://github.com/paboyle/Grid into develop 2024-03-12 15:16:24 +00:00
paboyle cf8632bbac Britney test option 2024-03-12 15:15:35 +00:00
portelli f17b8de907 fallback to _POSIX_HOST_NAME_MAX if HOST_NAME_MAX is not defined 2024-03-07 15:22:08 +09:00
Peter Boyle cc04dc42dc Merge branch 'develop' into feature/scidac-wp1 2024-03-06 14:55:21 -05:00
paboyle 1b93a9be88 Print out the hostname 2024-03-06 00:01:58 +00:00
Peter Boyle 839f9f1bbe Don't log memory by default 2024-01-17 16:25:50 -05:00
Peter Boyle 82fc4b1e94 Finalise 2023-11-23 18:19:41 -05:00
Peter Boyle b4f1740380 Finalise message 2023-11-23 18:19:16 -05:00
Peter Boyle b3925df9c3 Verbose on CPU-GPU xfer, remove performance by default 2023-10-24 10:25:01 -04:00
Peter Boyle afe10ba2a2 More digits 2023-10-20 19:27:13 -04:00
Chulwoo Jung 82c1ecf60f Block lanczos added 2022-11-30 16:08:40 -05:00
Peter Boyle f73db8f1f3 Synch clocks 2022-07-11 13:47:39 -04:00
Peter Boyle 63dbaeefaa Extra barrier prior to finalize just in case it fixes an issue on Tursa 2022-02-16 14:01:43 +00:00
Daniel Richtmann b4f8e87982 Have Grid's cli interface understand floats 2022-02-01 23:09:06 +01:00
Peter Boyle 68650b61fe Options controlling behaviour 2021-09-21 00:51:01 +02:00
Peter Boyle 5b3c530aa7 Return value 2021-08-23 15:30:45 +01:00
Peter Boyle c6a5499c8b Fail on non-apple 2021-08-22 18:40:55 +01:00
Peter Boyle ffbdd91e0e Apple happiness 2021-08-20 01:15:00 +01:00
Peter Boyle 15c50a7442 Explicit instantiate the template function 2021-03-18 15:40:42 -04:00
Christoph Lehner 4705aa541d Allow user to configure ShmDims via environment variables 2021-02-04 14:25:55 +01:00
Peter Boyle 1c673977fa Must ask for COMMMS_THREADS 2020-11-13 03:59:36 +01:00
Peter Boyle 37ec4b241c Default thread count sensible 2020-08-20 22:12:31 +02:00
Christoph Lehner 6365a89ba3 create separate InitMessage for MemoryManager that can be called after communicator setup 2020-07-30 07:25:05 -04:00
nmeyer-ur ea7f8fda5e fix typo 2020-07-22 09:34:05 +02:00
nmeyer-ur 906b78811b exit in Init when using --comms-overlap 2020-07-22 08:57:01 +02:00
Peter Boyle 577c064184 Memory manager initialise earlier 2020-06-24 08:24:38 -04:00
Peter Boyle b949cf6b12 PeekLocal needs a view to keep thread safe.
ALLOCATION_CACHEE reenable
2020-06-19 17:13:27 -04:00
Christoph Lehner 3dccd7aa2c Catch edge case in SharedMemoryMPI::GetShmDims; Change default units to consistent MB in init args; Want last element not past last element in MemoryManagerCache.cc 2020-06-14 13:26:01 -04:00
Peter Boyle f39c2a240b Priintinig and device memory size detection 2020-06-04 14:58:03 -04:00
Peter Boyle fb559614ad Initialise meemory manager 2020-06-03 09:12:47 -04:00
Peter Boyle ebb60330c9 Automatic data motion options beginning 2020-05-17 16:34:25 -04:00
Peter Boyle 07c0c02f8c Speed up Cshift 2020-05-11 17:02:01 -04:00
Peter Boyle 28a1fcaaff First compile against SYCL 2020-05-05 11:13:27 -07:00
Peter Boyle 0b3a3562c3 Some MPI (summit) create sigusr2, so trap that 2019-12-10 21:49:12 -05:00
Peter Boyle ec8e060ec7 Summit jsrun GPU mapping updates. Conffigure with --enable-jsrun 2019-10-31 11:46:09 -04:00
Peter Boyle 9cd33a7b9c Printing improvement 2019-07-31 08:01:24 +01:00
Peter Boyle 639dc1ab21 GPU reduction fix and also exit backtrace option 2019-07-31 01:23:23 +01:00