portelli/Grid
1
0
Fork 0
mirror of https://github.com/paboyle/Grid.git synced 2026-07-17 15:43:27 +01:00
Code Releases Activity
8,137 Commits 95 Branches 14 Tags
76c0ada1e1b4bd8999c77538fa219c1514f5bc4a
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
paboyle a3420e6fa9 Update for grid view logging 2025-08-14 21:29:20 +00:00
paboyle 732836d9f8 Missed one 2025-08-14 20:25:54 +00:00
paboyle 87658f7b53 ASSERT tripped in Shuhei's branch 2025-08-14 20:08:54 +00:00
paboyle e7f51e5fb1 Timer pointers for hadrons compat. 
Reluctantly, this interface is silly to pass timers around.
 2025-08-11 21:11:36 +01:00
paboyle 5adf2657dd Updated to compile and run fast on CUDA 2025-08-10 00:00:13 +01:00
paboyle 82cfff2990 A2A meson field BLAS based momentum project 2025-08-07 15:51:15 +00:00
paboyle 4397b1c442 Debugged momentum projection for A2A Meson Field 2025-08-07 15:51:01 +00:00
paboyle 9e6a4a4737 Assertion updates to macros (mostly) with backtrace. 
WIlson flow to include options for DBW2, Iwasaki, Symanzik.
View logging for data assurance
 2025-08-07 15:48:38 +00:00
paboyle 41f344bbd3 Merge with Christoph GPT checksum debug 2025-07-15 03:06:09 +00:00
paboyle a77cd50b2f Update comms logging in Cshift 2025-07-11 14:36:10 +00:00
paboyle 73af020f98 improved 2025-06-27 06:08:54 +00:00
paboyle bffb83c46e std::cout<<GridLogMessage<<"Debug:"<<std::endl; 
std::cout<<GridLogMessage<<"  --dylib-map     : print dynamic library map, useful for interpreting signal backtraces "<<std::endl;
    std::cout<<GridLogMessage<<"  --heartbeat     : periodic itimer wakeup (interrupts stuck system calls!) "<<std::endl;
    std::cout<<GridLogMessage<<"  --signal-delay n : pause for n seconds after signal handling (useful to get ALL nodes in stuck state) "<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-stdout  : print stdout from EVERY node to file Grid.stdout/err.rank "<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-signals : catch sigsegv and print a blame report, handle SIGHUP with a backtrace to stderr"<<std::endl;
    std::cout<<GridLogMessage<<"  --debug-heartbeat : periodically report backtrace "<<std::endl;

--dylib-map : Grid prints its dylib regions
--heartbeat : itimer based / SIGALRM wake up which seems to make Aurora
more stable
--debug-heartbeat : periodically report to stderr where we are in code

Now have libunwind option (configure: --with-unwind=<prefix>) to give an
Asynch-Signal safe backtrace. Avoid glibc backtrace due to mallocs.
 2025-06-27 06:08:54 +00:00
paboyle 7031f37350 Use libunwind for backtrace as it is signal asynch safe 2025-06-27 06:08:54 +00:00
paboyle 829dd74cb2 Verbose change 2025-06-27 06:08:54 +00:00
paboyle 66e671985d P2P 2025-06-27 06:08:54 +00:00
paboyle 5afcbcf0f3 Cshift uses flight recorder 2025-06-27 06:08:54 +00:00
paboyle 9730579312 Simplify and verbose 2025-06-27 06:08:51 +00:00
paboyle bfae14d035 More flight logging 2025-06-27 06:07:34 +00:00
paboyle b78fc73d19 Better signal handler 2025-06-27 06:07:34 +00:00
paboyle d418f78352 Making running on Aurora more debuggable 2025-05-23 20:58:16 +00:00
paboyle 25163998a0 Makes SYCL compiler happy 2025-05-23 20:57:11 +00:00
paboyle bb317aba8d Lattice = for sycl 2025-05-13 12:50:58 +00:00
paboyle 644cc6647e JSON update 2025-05-13 12:50:58 +00:00
paboyle 72397ce23b SYCL interface change 2025-05-13 12:50:58 +00:00
paboyle 25ab9325e7 Use hostVector but remove construct resize 2025-03-11 15:02:32 +00:00
paboyle 19f9378b98 Should work on Aurora nowb 2025-03-11 13:50:43 +00:00
paboyle 1d22841811 Working on aurora, GPT issue turned up is fixed 2025-03-06 03:20:18 +00:00
paboyle c4d9aa1a21 Config command that makes GPT happier 2025-02-27 20:12:49 +00:00
paboyle 6ae809ed40 Print not liked on GPT compile 2025-02-27 20:12:49 +00:00
paboyle 438dfbdb83 Only throw if there is a pending list entry in CommsComplete 2025-02-25 16:57:27 +00:00
paboyle b2ce760cf4 Verbose issue with GPT 2025-02-25 16:55:23 +00:00
paboyle ba9bbe0221 Bounce MPI through host 2025-02-12 19:34:59 +00:00
paboyle 4c3dd82d84 CSHIFT with bounce throuhgh Host memory on MPI packets 2025-02-12 19:09:53 +00:00
paboyle 44e911b5b7 Comment change 2025-02-12 17:37:55 +00:00
paboyle a7a16df9d0 GET not put has kinder barrier sequence for NVLINK type access as when 
GET is done, I can use it without barrier. Moves a barrier to a nicer
place, overlapped with DtoH DMA
 2025-02-12 14:59:28 +00:00
paboyle 382e0abefd Was issueing a double fence -- the gather also fences 2025-02-12 14:57:28 +00:00
paboyle 6fdefe5b90 Barrier sequencing if doing "GET" not "PUT" is different. 
This is somewhat better timing for Barriers
 2025-02-12 14:55:20 +00:00
paboyle 4788dd8e2e More states in packet progression for GPU non aware MPI 2025-02-12 14:53:57 +00:00
paboyle 1cc5f221f3 GET not put ordering is better as I know when I've got all MY data 2025-02-12 14:53:05 +00:00
paboyle 93251bfba0 GET not put for better ordering in the downstream dependent kernels -- I 
know when I'm done, so we can move a barrier / handshake between ranks
intranode to a point off critical path
 2025-02-12 14:50:21 +00:00
paboyle 18b79508b8 New line better for pretty print 2025-02-12 14:49:48 +00:00
paboyle 4de5ed1613 Remove vector view. The std::vector will not inform Memory manager of 
deletion and so a stale entry could be left. It is not and should not be
used.
 2025-02-12 14:48:46 +00:00
paboyle 0baaddbe98 Pipeline mode commit on Aurora. 5+ TF/s on 16^3x32 per tile at 384 
nodes.
More concurrency/fine grained scheduling is possible.
 2025-02-04 19:27:26 +00:00
paboyle b50fb34e71 Perf on Aurora 2025-02-01 18:39:34 +00:00
paboyle de84d730ff Fastest run config on Aurora to date 2025-02-01 18:08:40 +00:00
paboyle c4fc972fec Merge branch 'feature/deprecate-uvm' into develop 2025-01-31 16:32:36 +00:00
paboyle 8cf809e231 Best results on Aurora so far 2025-01-31 16:14:45 +00:00
paboyle 94019a922e Significantly better performance on Aurora without using pipeline mode 2025-01-30 16:36:46 +00:00
paboyle d6b2727f86 Pipeline mode getting better -- 2 nodes @ 10TF/s per node on Aurora 2025-01-29 09:22:21 +00:00
paboyle 74a4f43946 Optional host buffer bounce for no CUDA aware MPI 2025-01-28 15:22:46 +00:00
paboyle 1caf8b0f86 Rename 2025-01-28 15:22:37 +00:00
paboyle 8fe429346f Dslash testing for reproduce 2024-11-11 23:11:11 +00:00
paboyle 5cc4f3241d Meson field test 2024-10-18 15:42:30 +00:00
paboyle a78a61d76f Update configure 2024-10-15 14:38:45 +00:00
paboyle 2eff3f34ed Alternate reduction; default to grids own but make a configure flag 
--enable-reduction=grid|mpi
 2024-10-15 14:36:06 +00:00
paboyle 03687c1d62 Final version of test, closer to original again 2024-10-15 14:35:17 +00:00
paboyle febfe4e77f Make my own reduction a configure flag 2024-10-15 14:32:35 +00:00
paboyle 4d1aa134b5 Use normal reduction, configure flag to force deterministic 2024-10-15 14:32:11 +00:00
paboyle 5ec879860a Odd rounding issue - bears looking into 2024-10-15 14:30:54 +00:00
paboyle b728af903c Fast axpy norm under CFLAG 2024-10-11 03:23:09 +00:00
paboyle 54f1999030 axpy_norm_fast -- wasn't using the determinstic MPI sum causing issues 2024-10-11 03:22:18 +00:00
paboyle fd58f0b669 Return ok 2024-10-11 03:21:21 +00:00
paboyle c5c67b706e cl::sycl -> SYCL 2024-10-10 22:04:12 +00:00
paboyle be7a543e2c Revert barriers -- these were not the problem 2024-10-10 22:03:29 +00:00
paboyle 68f112d576 New software moves cl::sycl 2024-10-10 22:03:04 +00:00
paboyle ec1395a304 Better flight logging 2024-10-10 22:01:57 +00:00
paboyle beb0e474ee Use deterministic own brand reduction 2024-10-10 22:01:24 +00:00
paboyle 2b5fdcbbc5 New software version 2024-10-10 21:59:02 +00:00
paboyle 295127d456 Deterministic homebrew reduction 2024-10-10 21:58:26 +00:00
paboyle 7dcfb13694 New software stack 2024-10-10 21:57:35 +00:00
paboyle 9fa8bd6438 Configure for AOT on Aurora latest software 2024-09-23 11:25:44 +00:00
paboyle 02c8178f16 Almost working on Aurora 2024-09-23 09:43:50 +00:00
paboyle e637fbacae Verbose remove 2024-09-23 09:42:43 +00:00
paboyle 066544281f Deprecate UVM 2024-09-17 13:34:27 +00:00
paboyle 11be10d2c0 Aurora testing 2024-09-10 18:11:52 +00:00
paboyle 160969a758 UVM tester, doesn't turn up anything 2024-09-10 18:09:42 +00:00
paboyle 622f78ebea SYCL updates -- operator = giving trouble on Aurora. 
SYCL reduction is failing intermittently with SVM interface - returns
zero, expect non-zero.
Think I need to remove ALL dependence on SVM.
 2024-09-04 13:53:48 +00:00
paboyle 89c0519f83 Repro test 2024-03-12 16:11:33 +00:00
paboyle 2704b82084 Merge branch 'develop' of https://github.com/paboyle/Grid into develop 2024-03-12 15:16:24 +00:00
paboyle cf8632bbac Britney test option 2024-03-12 15:15:35 +00:00
paboyle d224297972 PBS scripts 2024-03-12 15:15:16 +00:00
paboyle b812a7b4c6 Staggered launch script 2024-03-06 01:32:40 +00:00
paboyle 891a366f73 Repro CG script 2024-03-06 01:22:55 +00:00
paboyle 10116b3be8 Force device copyable and tell SYCL to shut it. 2024-03-06 01:13:27 +00:00
paboyle a46a0f0882 force device copyable and don't take crap from SYCL 2024-03-06 01:12:49 +00:00
paboyle a26a8a38f4 Merge branch 'develop' of https://github.com/paboyle/Grid into develop 2024-03-06 00:05:00 +00:00
paboyle 7435315d50 More blasted shell variables 2024-03-06 00:03:59 +00:00
paboyle 9b5f741e85 Reproducing CG can be more useful now 2024-03-06 00:03:16 +00:00
paboyle 517822fdd2 SPR HBM benchmarking right and also PVC batched GEMM 2024-03-06 00:02:27 +00:00
paboyle 1b93a9be88 Print out the hostname 2024-03-06 00:01:58 +00:00
paboyle 783a66b348 Deterministic reduction please 2024-03-06 00:01:37 +00:00
paboyle 976c3e9b59 Hack for flight logging CG inner products. 
Can be made to work, but could put in some more serious infrastructure
for repro testing and blame attribution (Britney test) if necessary
 2024-03-05 23:59:57 +00:00
paboyle f8ca971dae Use of a bare PRECISION macro is not namespace safe and collides with 
SYCL
 2024-03-05 23:59:13 +00:00
paboyle 21bc8c24df OneMKL batched blas starting 2024-03-05 23:58:20 +00:00
paboyle 30228214f7 SYCL conflict with Eigen 2024-03-05 23:56:10 +00:00
paboyle 73c0b29535 Merge branch 'develop' of https://github.com/paboyle/Grid into develop 2024-02-13 20:19:32 +00:00
paboyle 303b83cdb8 Scaling benchmarks, verbosity and MPICH aware in acceleratorInit() 
For some reason Dirichlet benchmark fails on several nodes; need to
debug this.
 2024-02-13 19:48:03 +00:00
paboyle 5ef4da3f29 Silence verbose 2024-02-13 19:47:36 +00:00
paboyle 1502860004 Benchmark scripts 2024-02-13 19:47:02 +00:00
paboyle 585efc6f3f More benchmark scripts 2024-02-13 19:40:49 +00:00