Commit Graph
8441 Commits
Author SHA1 Message Date
Peter Boyle 21b53c06d1 FI investigatins 2026-08-28 23:31:54 -04:00
Peter Boyle e03797e882 More FI_MR related 2026-08-28 22:01:36 -04:00
Peter Boyle 43c6573ca4 Memory manager update to drop cache 2026-08-28 17:03:41 -04:00
Peter Boyle f367bc8bce Solving memory pressure in NRHS=12 solver 2026-08-28 13:52:45 -04:00
Peter Boyle 63c2cdb712 Debug ulimit as core files driving me crazy 2026-08-28 11:20:43 -04:00
Peter Boyle 9b4c388585 Update for latt parameter; claude's predeliction for using shell variables rather than "grid" style command line args is getting annoying 2026-08-28 10:04:36 -04:00
Peter Boyle 576cebd56a Job for debug 2026-08-28 09:03:56 -04:00
Peter Boyle 86da22fb78 Debug page fault 2026-08-28 08:52:11 -04:00
Peter Boyle 27e32b1633 Stop mucking about with THREAD_MULTIPLE 2026-08-28 07:06:45 -04:00
Peter Boyle 2264afda84 Allreduce benchmarking 2026-08-27 22:11:06 -04:00
Peter Boyle 792d2a8258 Masked/orthog dim CartesianRingAllReduce and CartesianRingAllGather 2026-08-27 17:10:49 -04:00
Peter Boyle 7abc19dc03 Allow subsets of dimensions for spatial only sums 2026-08-27 17:03:14 -04:00
Peter Boyle 074bddd48b Probing environment effects on dense inversion time 2026-08-27 13:56:02 -04:00
Peter Boyle 6cea81bc2c More probing why test harness vs. slate was a little faster running Grid's dense inverse than normal environment 2026-08-27 13:42:18 -04:00
Peter Boyle 68b9fa86d7 Faster leaf inverse 2026-08-27 12:53:26 -04:00
Peter Boyle d44964d923 Faster dense inverse, I hope 2026-08-27 12:13:46 -04:00
Peter Boyle 52998539bc Bigger leaf operations should spead up distributed inverse 2026-08-27 12:08:37 -04:00
Peter Boyle 88acdb8172 More perf diagnostics 2026-08-27 11:22:35 -04:00
Peter Boyle 0857f3aa82 Mor ecomms probing 2026-08-27 09:26:24 -04:00
Peter BoyleandClaude Fable 5 0979371c45 SUMMA telemetry: ring time histogram by message size
Per log2-size bucket: messages, GB, seconds inside SendToRecvFrom, GB/s,
% of ring time.  Decomposes the 2 GB/s/rank average (probe: 11-20 GB/s
at 8 MB) into latency-bound small messages vs slow large ones vs wait.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAhdQHrkzKmzfxpxW1whLn
2026-08-27 02:02:30 -04:00
Peter BoyleandClaude Fable 5 96071aa492 Dense allgather: stage/scatter x by global-site order, not rank-major block
DENSE_DEVICE_SUM=4 assumed each rank's rows of x were the contiguous
[me*nrows,...) block; x and the slab columns are in global-site order
(hX[myGsite*nbasis+b]), the contiguous block is the rank-major index.
Coincide on one rank only -> Frontier VERIFY 0.9965.  Stage via myGsite,
scatter through the inverse of BuildRankMajorMap.  4-rank laptop VERIFY
7.38e-07, identical to the allreduce path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAhdQHrkzKmzfxpxW1whLn
2026-08-27 02:02:01 -04:00
Peter Boyle e5f1553232 Better reduction 2026-08-26 23:36:31 -04:00
Peter Boyle 06c6011229 Timestamp improvement 2026-08-26 21:56:04 -04:00
Peter Boyle cbe97cc40d Smoother search improvement 2026-08-26 21:37:36 -04:00
Peter Boyle dc1ae3185a Ring allgather too. Let's nail the Dense CoarseCoarse 2026-08-26 21:05:36 -04:00
Peter Boyle 8e4a304626 Less aggressive L5 2026-08-26 20:17:19 -04:00
Peter Boyle 5372dce0c2 Rerecord, not fade away ! 2026-08-26 20:15:14 -04:00
Peter Boyle bc67e2c584 abs thrust issue 2026-08-26 17:58:18 -04:00
Peter Boyle e15099eb7b Both changes to improve the buffer persistence in teh dense distributed inverse and to add verbosity to polynomial smoothers 2026-08-26 17:48:03 -04:00
Peter Boyle 5f8f396e9b More jobs for testing Mgrid 2026-08-26 14:22:46 -04:00
Peter Boyle 841e59e8c0 GCR coefficient recording and playback 2026-08-26 14:22:00 -04:00
Peter Boyle b55ca3d50d Consolidate polynomial smoothers + logging time breakdown in Dense Coarsest invers 2026-08-26 14:20:50 -04:00
Peter Boyle d7c9076228 Power method added 2026-08-26 06:54:20 -04:00
Peter Boyle 7a292bc4c6 Slate compile fix 2026-08-25 21:39:26 -04:00
Peter Boyle 9213f6e533 Improvements for mulktigrid. + Schur setup improvements 2026-08-25 21:37:12 -04:00
Peter Boyle 5685203b7b Frontier jobs 2026-08-25 19:18:16 -04:00
Peter Boyle 22dda4a8ed Update 2026-08-25 19:17:59 -04:00
Peter Boyle eb22145f49 Updated slate 2026-08-25 17:27:37 -04:00
Peter Boyle 5d92f56bed omp issues combined with hipcc 2026-08-25 15:45:49 -04:00
Peter Boyle d42d1855c2 SLATE build updates 2026-08-25 15:33:07 -04:00
Peter Boyle 0a81a41729 Slate changes and warn suppression 2026-08-25 15:29:21 -04:00
Peter Boyle 229f4808c7 Ring all reduce first cut 2026-08-25 12:53:24 -04:00
Peter Boyle 167753aa3f Adding ring all reduce for large vectors 2026-08-25 12:07:11 -04:00
Peter Boyle dbbd5dba35 Potentially faster GCR linalg 2026-08-25 07:29:02 -04:00
Peter Boyle 33757c3c94 FOrmat warn avoidance 2026-08-24 20:32:54 -04:00
Peter Boyle 7438e56f94 More controls 2026-08-24 19:42:38 -04:00
Peter Boyle 47416230b1 Verbose on finalize improvement 2026-08-24 19:42:38 -04:00
Peter Boyle 6a854ebd0d Print trace for teh GCR step 2026-08-24 19:42:38 -04:00
Peter Boyle 8c6bcddbab Coefficent of GCR printing for poly recording 2026-08-24 19:40:27 -04:00
Peter Boyle effa31e761 Drop redundant set zero's and optimise persistent history vectors for smoothers 2026-08-24 15:48:26 -04:00