1
0
mirror of https://github.com/paboyle/Grid.git synced 2024-11-14 01:35:36 +00:00
Commit Graph

529 Commits

Author SHA1 Message Date
c698b16d75 function to generate Chroma-style gamma matrix products 2016-05-01 18:30:35 -07:00
paboyle
5341977948 IMCI fixes. Thought I had committed these. The "real" disambiguation
between std::real and Grid::real shouldn't have been necessary and I don't
know why only the icpc v16.0 on babbage hits it.
May need a longer term rename of Grid::real or some careful EnableIf work.
2016-04-30 03:34:16 -07:00
f6c53e5039 Merge commit '1e554350acae0e67fa7177ed0db9d4f684a54af2' 2016-04-30 00:17:52 -07:00
6aa000176f Fermion <-> Propagator functions 2016-04-30 00:14:33 -07:00
paboyle
1e554350ac The threaded coms didn't agree with GCC. Suprised, and looks like GCC bug. 2016-04-29 16:49:18 -07:00
paboyle
c79ea0dcef Fixingn IMCI 2016-04-22 21:52:54 -07:00
paboyle
8fd8bc25e9 simd 5th dim with rotation 2016-04-19 15:39:00 -07:00
paboyle
ba427abde9 simd 5d 2016-04-19 15:38:39 -07:00
paboyle
9b6ab6db16 simd in 5th dimension support 2016-04-19 15:38:01 -07:00
paboyle
806a83d38b simd in fifth dim support for dwf 2016-04-19 15:36:19 -07:00
neo
339be37dba Debugging smeared HMC 2016-04-13 17:00:14 +09:00
neo
a87b744621 HMC runs but does not accept with smearing on 2016-04-07 16:45:11 +09:00
paboyle
b1192a8908 Benchmark_zmm added 2016-04-06 03:00:07 -07:00
paboyle
e8dddb1596 Adding extra benchmark 2016-04-06 10:32:54 +01:00
97d0d56bcb Debugging Smearing routines (set_fj) 2016-04-06 17:58:43 +09:00
7c7ea35ffb Putting the Traceless Antihermitian part outside the deriv in pseudofermion actions 2016-04-05 16:28:09 +09:00
4b1cf580e0 Debugging the Smearing routines 2016-04-05 16:19:30 +09:00
paboyle
e67fc2be18 Adding a trial for openmp overhead minimisation 2016-03-31 16:00:37 +01:00
paboyle
8052556275 Cleaning up the single/double kernel implementation switch 2016-03-31 14:51:32 +01:00
paboyle
60d965f79e AVX512 improvements; sigfpe trapping too 2016-03-30 08:42:34 +01:00
paboyle
1ecbf9794d Merge branch 'master' of https://github.com/paboyle/Grid 2016-03-30 08:37:55 +01:00
paboyle
c77b7ee897 AddSub based alternate SU3 routine 2016-03-28 17:55:22 -06:00
paboyle
1e355a51e1 Interface change 2016-03-27 23:46:55 -07:00
paboyle
21abaf7e91 Gamma sign change 2016-03-28 00:35:45 -06:00
paboyle
165bffc2e7 Avx512 changes for assembler kernels 2016-03-26 22:25:45 -06:00
paboyle
644fd6d32e Build avx512 clean 2016-03-25 09:35:33 -07:00
paboyle
60d4564151 ICC no compile fix 2016-03-16 02:30:40 -07:00
paboyle
090e7aa930 Merge remote-tracking branch 'origin/chulwoo-dec12-2015'
Merge Chulwoo's Lanczos related improvements.
Merge Nd!=4 fixes for pure gauge HMC from Evan.
2016-03-08 09:55:14 +00:00
paboyle
325e745daa Merge branch 'master' of https://github.com/paboyle/Grid 2016-03-02 07:04:03 -08:00
paboyle
61413565d0 Back off the inlined spin proj as not working 2016-03-02 07:03:09 -08:00
2d8bb356e3 Smearing routines compile (still untested) 2016-02-25 02:43:59 +09:00
a7251f28c7 Stout smearing compiles (untested) 2016-02-24 03:16:50 +09:00
Antonin Portelli
497e7e4c53 BG/Q compatibility fix 2016-02-23 15:57:38 +00:00
Peter Boyle
6aeaf6f568 Parallel IO worked on. I'm puzzled because I already thought I shook this out on MacOS + OpenMPI and then
turned up problems on the BlueWaters Cray.

Gets 75MB/s from home filesystem on parallel configuration read. Need to make the RNG IO parallel,
and also to look at aggregating bigger writes for the parallel write.
Not sure what the home filesystem is.
2016-02-21 08:03:21 -06:00
Peter Boyle
40f2db9bc0 Disable metropolis step until 10 traj covered. Should move to exposing these
in XML input and start having "applications" directory.
2016-02-21 08:01:44 -06:00
Jung
9f0d9ade68 Added configure flag for LAPACK. Tested ImplicitlyRestartedLanczos::calc()
Checking in before cleaning up
2016-02-20 02:50:32 -05:00
neo
c1b1b89d17 More on smearing routines, writing APEsmear (dev) 2016-02-19 17:15:27 +09:00
neo
771235017d Adding smearing routines (development) 2016-02-19 15:30:41 +09:00
paboyle
3425751cb8 Missing return value 2016-02-19 01:06:03 +00:00
Peter Boyle
22422a84d9 Small problem in compressor fix 2016-02-17 19:03:09 -06:00
Peter Boyle
c9fadf97a5 Simplify the compressor interface again. 2016-02-17 18:16:45 -06:00
Peter Boyle
81395e85d1 Regressing to not overlap comms and compute becasue bluewaters, edison, and cori are so rubbish at it. 2016-02-16 13:56:44 -06:00
Peter Boyle
a0fc47c6f9 Cheaper implementation 2016-02-15 16:02:36 -06:00
paboyle
e2f73e3ead Updates for shmem 2016-02-10 16:50:32 -08:00
neo
6371676a75 Correcting some compilation errors for clang-sse 2016-02-10 11:37:03 +09:00
Jung
bd84c23298 definitions reconciled. 2016-01-25 16:30:59 -05:00
Jung
7aa8d5e8af Faiing to compile, comparing with master 2016-01-25 16:03:02 -05:00
Jung
6012b0ec23 Checking in changes before changing to chulwoo-dec12-2015 2016-01-25 09:40:58 -05:00
Jung
411ac49dd7 GparityWilsonTM typedef added. Not yet tested
Conflicts:
	configure
	lib/qcd/action/fermion/WilsonKernels.h
2016-01-25 01:36:28 -05:00
Jung
5c57d4f403 Merge branch 'master' of https://github.com/paboyle/Grid into scidac1_2
Conflicts:
	lib/qcd/action/fermion/WilsonKernels.h
2016-01-11 11:36:45 -05:00
paboyle
fc6ad65751 Pushed the overlap comms tweaks 2016-01-11 06:34:22 -08:00
paboyle
dafc74020c Overlap comms compute improvements in hand op kernels, and better timing from Edison and Cori 2016-01-10 16:54:27 -08:00
paboyle
d19321dfde Overlap comms compute changes 2016-01-10 19:20:16 +00:00
Jung
5924e5a562 Merge branch 'master' of https://github.com/paboyle/Grid into scidac1_2
Conflicts:
	configure
	lib/qcd/action/Actions.h
	lib/qcd/action/fermion/WilsonKernels.h
2016-01-06 03:44:57 -05:00
paboyle
c99d748da6 Timing reports in benchmarks now reflect the asynch comms thread statistics 2016-01-04 14:42:16 +00:00
paboyle
02452afd36 Optional overlap of comms with compute 2016-01-04 14:18:40 +00:00
paboyle
331768dcff Added overlap comms compute mode 2016-01-03 01:38:11 +00:00
paboyle
aae8bf31a7 Global edit adding copyright and license info to every source file. 2016-01-02 14:51:32 +00:00
paboyle
1e68b1c1bd Create a benign default for gparity twists 2016-01-02 14:06:53 +00:00
paboyle
5a80930dd2 Charge conjugation boundary conditions for gauge fields implemented as a policy
class, changing the nature of covariant Cshifts used in
plaquettes, rectangles and staples.

As a result same code is used for the plaq and rect action independent of the BC type.

Should probably isolate the BC in a separate class that Gimpl takes as a template param.
Do the same with smearing policies.

This would then allow composition of BC with smearing etc....
2016-01-02 13:37:25 +00:00
paboyle
841a37f941 Fix to WilsonCompressor that fixes a bug in comms phase due to the sign change on gamma
matrix in hopping term.
Add logging of time spent in CG.
2015-12-29 23:49:41 +00:00
Azusa Yamaguchi
e6cad3821c Logging improvement 2015-12-29 19:51:18 +00:00
Azusa Yamaguchi
98de1cbb6a Optimised version of rectangle term staples.
~3.4x faster than the naive.
2015-12-29 19:22:59 +00:00
Azusa Yamaguchi
f7d61b8b81 Plaq plus rectangle and Iwasaki, Symanzik DBW2.
http://arxiv.org/pdf/hep-lat/0610075.pdf plaq and rect regress plausibly over 100 trajectories
and under HMC with average plaq and rectangle coming out ok.
2015-12-28 16:39:26 +00:00
Azusa Yamaguchi
78c4e862ef Plaq, Rectangle, Iwasaki, Symanzik and DBW2 workign and HMC regresses to http://arxiv.org/pdf/hep-lat/0610075.pdf 2015-12-28 16:38:31 +00:00
paboyle
0afcf1cf13 Moved all the HMC tests over to using a single HmcRunner class that manages checkpoint strategies and such like 2015-12-22 11:19:25 +00:00
paboyle
08edbb5cbe HMC bit repro across checkpoints. Fixed parallel RNG issue with threading.
Conclusion: c++11 distributions not thread safe and must us distinct dist as well as distinct engine
per site. Makes sense when you think of box muller. Also added a reset of dist on fill to ensure
repro across checkpoints.
2015-12-22 08:54:40 +00:00
paboyle
0abfbcc8eb Naming of files improvement. 2015-12-21 15:37:26 +00:00
paboyle
1b94253ba4 Logging improvement 2015-12-21 15:36:28 +00:00
paboyle
36e6f9ac7b Bug fix. Guess not initialised in refresh step; didn't hit before due to luck in not having a vector
created with NAN data.
2015-12-21 15:34:35 +00:00
paboyle
2f41691c11 Bug fix. Guess was not zeroed prior to CG call. Was earlier accidentally benign just due to luck. 2015-12-21 15:33:36 +00:00
paboyle
31ca609d12 HMC checkpointing .
Need a general HMC framework to work in restart.
2015-12-20 02:29:51 +00:00
paboyle
e108e708a3 Wilson TM tests and compiles in 2015-12-17 23:06:33 +00:00
paboyle
67ccb043f1 Added TM fermions for DSDR etc.. 2015-12-17 22:34:28 +00:00
Jung
eb1759d7ea Added Gparity instantiation to no HANDOPT case
deleted configure (as intended?)
2015-12-16 00:04:09 -05:00
paboyle
34a0fde2ad Fixes to fermion force terms after sign of gamma_mu (0...3) change.
Thought I had already committed these.

Believe I have got the Gparity fermion force working.

* tests/Test_gpdwf_force.cc     -- correctly predicts dS for two flavour pseudofermion
                                   based on a small dt update of U field.

* tests/Test_hmc_EODWFRatio_Gparity.cc -- ran 1 trajectory on 8^4 with dH=0.21.

Need to accumulate a full plaquette log to believe fully which will take some hours of run time.
2015-12-15 23:14:12 +00:00
Jung
bc34b7e808 Merge branch 'master' of https://github.com/paboyle/Grid into scidac1_2
Conflicts:
	lib/qcd/action/fermion/WilsonKernels.h
	tests/Make.inc
2015-12-15 11:11:59 -05:00
Jung
284453c5e9 Added gparity mobius defs, added params to ScaledShamir
checking in before puling master
2015-12-14 12:15:06 -05:00
paboyle
3ce10aa975 Fix a regression failure on Mobius; chroma regression added 2015-12-10 22:55:00 +00:00
Jung
f2b4edc090 Fixes for Gparity comparison with CPS (Instantiation, Gamma matrix convention) 2015-12-07 02:04:57 -05:00
paboyle
b2c02a6106 Runs fastst on cori 2015-11-28 16:58:16 -08:00
paboyle
e9ff25b06b Small threading change makes a difference on Cori. 2015-11-07 00:07:05 -08:00
paboyle
05a7029600 Stencil change 2015-11-07 00:06:31 -08:00
paboyle
899ca41cb8 Merge branch 'master' of github.com:paboyle/Grid
Conflicts:
	lib/qcd/action/fermion/WilsonFermion5D.cc
2015-11-06 03:50:04 -08:00
paboyle
d29b4c1dee Assembler files 2015-11-06 03:48:48 -08:00
paboyle
a2ff068e29 Asm and threading for many core 2015-11-06 03:47:14 -08:00
paboyle
17af18dcab Changes for AVX512 assembler 2015-11-06 03:45:51 -08:00
Peter Boyle
28022755ae Stencil class name global change to StencilImpl typedef 2015-11-06 05:30:17 -06:00
paboyle
1159de165c Asm option for AVX512 2015-11-05 22:04:51 -08:00
paboyle
16c7993434 Merge branch 'master' of github.com:paboyle/Grid
Conflicts:
	lib/simd/Grid_avx512.h
	lib/simd/Grid_imci.h
2015-11-04 03:32:10 -08:00
paboyle
4e65ad21ac Adding a routine for AVX512 / IMCI with explicit assembly implementations 2015-11-04 03:15:08 -08:00
Peter Boyle
abb23df83f formatting only 2015-11-04 10:00:27 +00:00
paboyle
1878bf97d0 Babbage fix 2015-09-30 16:04:01 -07:00
paboyle
a660ce716b No compile babbage fix 2015-09-30 16:02:44 -07:00
Peter Boyle
64d64d1ab6 Updating to modify non-inlining permute routines and hopefully get better reg use and
enhance performance.
2015-09-25 08:55:04 -07:00
Peter Boyle
5ef42add2d Changes to remove warnings under icc; disambiguate AVX512 from IMCI correctly
and drop swizzles in AVX512. Don't know why these compiled.
2015-09-23 05:23:45 -07:00
Peter Boyle
2f38ebc446 Reintroducing the hand unrolled loops 2015-09-08 17:45:30 +01:00
Peter Boyle
638d6675ee Tested rms dH is ~ dt^4 numerically, so believe the ForceGradient is correct now.
Paranoia makes me want to diddle with the FG step to ensure dt^2 reappears.
2015-08-31 16:33:20 +01:00
Peter Boyle
357c6ab46d Reunitarise. Complete the HMC and integrator changes. 2015-08-31 16:32:04 +01:00
Peter Boyle
755dca9533 Added ForceGradient integrator. dH dropped so seems to work. Will only
believe it is right once I have pulled a dt^4 error scaling plot out.
2015-08-31 06:23:02 +01:00
Peter Boyle
29fd004d54 Unified integrator and integrator algorithm into virtual class used as a policy for the
HMC.
2015-08-30 13:39:19 +01:00
Peter Boyle
aa52fdadcc Global edit on HMC sector -- making GaugeField a template parameter and
preparing to pass integrator, smearing, bc's as policy classes to hmc.

Propose to unify "integrator" and integrator algorithm in a base/derived
way to override step. Want to read through ForceGradient to ensure
that abstraction covers the force gradient case.
2015-08-30 12:18:34 +01:00
Peter Boyle
76d752585b Started a tidy up in the HMC sector. Now comfortable with the two level integrators;
to a little figure out what Guido had done & why -- but there is a neat saving of force
evaluations across the nesting time boundary making use of linearity of the leapP in dt.

I cleaned up the printing, reduced the volume of code, in the process sharing printing
between all integrators. Placed an assert that the total integration time for all integrators
must match at end of trajectory.

Have now verified e-dH = 1 for nested integrators in Wilson/Wilson runs with both
Omelyan and with Leapfrog so substantial confidence gained.
2015-08-29 17:18:43 +01:00
Peter Boyle
dc814f30da Binary IO file for generic Grid array parallel I/O.
Number of IO MPI tasks can be varied by selecting which
dimensions use parallel IO and which dimensions use Serial send to boss
I/O.

Thus can neck down from, say 1024 nodes = 4x4x8x8 to {1,8,32,64,128,256,1024} nodes
doing the I/O.

Interpolates nicely between ALL nodes write their data, a single boss per time-plane
in processor space [old UKQCD fortran code did this], and a single node doing all I/O.

Not sure I have the transfer sizes big enough and am not overly convinced fstream
is guaranteed to not give buffer inconsistencies unless I set streambuf size to zero.

Practically it has worked on 8 tasks, 2x1x2x2 writing /cloning NERSC configurations
on my MacOS + OpenMPI and Clang environment.

It is VERY easy to switch to pwrite at a later date, and also easy to send x-strips around from
each node in order to gather bigger chunks at the syscall level.

That would push us up to the circa 8x 18*4*8 == 4KB size write chunk, and by taking, say, x/y non
parallel we get to 16MB contiguous chunks written in multi 4KB transactions
per IOnode in 64^3 lattices for configuration I/O.

I suspect this is fine for system performance.
2015-08-26 13:40:29 +01:00
Peter Boyle
e8d63c9178 Merge branch 'master' of https://github.com/paboyle/Grid 2015-08-19 05:49:00 +01:00
Peter Boyle
c54c086f17 Even odd preconditioned one flavour ratio
(no support for non-const EE schur block)
2015-08-19 05:46:58 +01:00
Peter Boyle
dd6bb73ee0 Added one flavour rational ratios (unprec) 2015-08-19 04:58:40 +01:00
Peter Boyle
fc160eeccc Added one flavour rational ratios (unprec) 2015-08-19 04:58:40 +01:00
Peter Boyle
48db72259e EvenOdd schur decomposed mpcdagmpc version of rhmc determinant.
dH is also small and plaquette looks right.
2015-08-18 18:37:39 +01:00
Peter Boyle
570150f1d3 EvenOdd schur decomposed mpcdagmpc version of rhmc determinant.
dH is also small and plaquette looks right.
2015-08-18 18:37:39 +01:00
Peter Boyle
5c364f8082 One flavour rational unprec added; untested but does compile.
Moving param structs into a single header for later connection to file I/O using
macromagic.h
2015-08-18 14:40:08 +01:00
Peter Boyle
a842a6c94d One flavour rational unprec added; untested but does compile.
Moving param structs into a single header for later connection to file I/O using
macromagic.h
2015-08-18 14:40:08 +01:00
Peter Boyle
bdcbfe9310 Even Odd two flavour ratio added and dH == small 2015-08-18 10:37:08 +01:00
Peter Boyle
9306921ded Even Odd two flavour ratio added and dH == small 2015-08-18 10:37:08 +01:00
Peter Boyle
76f3855629 Merge branch 'master' of https://github.com/paboyle/Grid 2015-08-18 09:23:58 +01:00
Peter Boyle
8621e2409f Merge branch 'master' of https://github.com/paboyle/Grid 2015-08-18 09:23:58 +01:00
Peter Boyle
6212807a77 Small dh obtained in two flavour ratio so looks ok. 2015-08-18 09:21:29 +01:00
Peter Boyle
7622f0c441 Small dh obtained in two flavour ratio so looks ok. 2015-08-18 09:21:29 +01:00
Peter Boyle
0bc38a69ce Adding PV pseudofermion in prep for DWF HMC.
Not compiled this yet, but cloned in from BFM.
2015-08-18 09:19:42 +01:00
Peter Boyle
25d0eae50c Adding PV pseudofermion in prep for DWF HMC.
Not compiled this yet, but cloned in from BFM.
2015-08-18 09:19:42 +01:00
Peter Boyle
24382d77bb Adding PV pseudofermion in prep for DWF HMC.
Not compiled this yet, but cloned in from BFM.
2015-08-17 23:14:48 +01:00
Peter Boyle
ef6a9e6b07 Adding PV pseudofermion in prep for DWF HMC.
Not compiled this yet, but cloned in from BFM.
2015-08-17 23:14:48 +01:00
Peter Boyle
353d66def1 Unused apparently 2015-08-16 01:41:05 +01:00
Peter Boyle
b8166af92b Unused apparently 2015-08-16 01:41:05 +01:00
Peter Boyle
afeabe0d23 Tidying 2015-08-16 00:14:10 +01:00
Peter Boyle
6180487517 Tidying 2015-08-16 00:14:10 +01:00
Peter Boyle
53da927c3c Merge branch 'master' of https://github.com/paboyle/Grid 2015-08-15 23:59:04 +01:00
Peter Boyle
f0e32f12cf Merge branch 'master' of https://github.com/paboyle/Grid 2015-08-15 23:59:04 +01:00
Peter Boyle
155c164b0c * Finished the template/policy style introduction of gparity, except the gparity force terms.
So valence sector looks ok.

FermionOperatorImpl.h provides the policy classes.

Expect HMC will introduce a smearing policy and a fermion representation change policy template
param. Will also probably need multi-precision work.

* HMC is running even-odd and non-checkerboarded (checked 4^4 wilson fermion/wilson gauge).

There appears to be a bug in the multi-level integrator -- <e-dH> passes with single level but
not with multi-level.

In any case there looks to be quite a bit to clean up.

This is the "const det" style implementation that is not appropriate  yet for clover since
it assumes that Mee is indept of the gauge fields. Easily fixed in future.
2015-08-15 23:25:49 +01:00
Peter Boyle
55cfc89459 * Finished the template/policy style introduction of gparity, except the gparity force terms.
So valence sector looks ok.

FermionOperatorImpl.h provides the policy classes.

Expect HMC will introduce a smearing policy and a fermion representation change policy template
param. Will also probably need multi-precision work.

* HMC is running even-odd and non-checkerboarded (checked 4^4 wilson fermion/wilson gauge).

There appears to be a bug in the multi-level integrator -- <e-dH> passes with single level but
not with multi-level.

In any case there looks to be quite a bit to clean up.

This is the "const det" style implementation that is not appropriate  yet for clover since
it assumes that Mee is indept of the gauge fields. Easily fixed in future.
2015-08-15 23:25:49 +01:00
Peter Boyle
f40475f382 Reorganising the Fermion interface 2015-08-14 14:16:45 +01:00
Peter Boyle
ba8c09a58e Reorganising the Fermion interface 2015-08-14 14:16:45 +01:00
Peter Boyle
cc63078de5 Gparity works now even if simd distributed in a Gparity twist direction.
Tested by doubling lattice in t-direction.
2015-08-14 12:57:42 +01:00
Peter Boyle
59d66eb17a Gparity works now even if simd distributed in a Gparity twist direction.
Tested by doubling lattice in t-direction.
2015-08-14 12:57:42 +01:00
Peter Boyle
4dc7c36aa8 Gparity works now even if simd distributed in a Gparity twist direction.
Tested by doubling lattice in t-direction.
2015-08-14 12:57:42 +01:00
Peter Boyle
e6bed000c3 Gparity valence test now working.
Interface in FermionOperator will change a lot in future
2015-08-14 00:01:04 +01:00
Peter Boyle
028e2061e0 Gparity valence test now working.
Interface in FermionOperator will change a lot in future
2015-08-14 00:01:04 +01:00
Peter Boyle
7d3512ab21 Gparity valence test now working.
Interface in FermionOperator will change a lot in future
2015-08-14 00:01:04 +01:00
Peter Boyle
fc9b36c769 Gamma5 mult direct 2015-08-13 10:51:29 +01:00
Peter Boyle
2c216a42f9 Gamma5 mult direct 2015-08-13 10:51:29 +01:00
Peter Boyle
45b01858a8 Gamma5 mult direct 2015-08-13 10:51:29 +01:00
Peter Boyle
c39078162e Gparity improvements 2015-08-13 10:51:01 +01:00
Peter Boyle
145b807ba2 Gparity improvements 2015-08-13 10:51:01 +01:00
Peter Boyle
1c2d148bfa Gparity improvements 2015-08-13 10:51:01 +01:00
Peter Boyle
7e9203d8e0 Some bug fixes for more complicated types introduced with gparity 2015-08-13 10:50:34 +01:00
Peter Boyle
8d4c43327b Some bug fixes for more complicated types introduced with gparity 2015-08-13 10:50:34 +01:00
Peter Boyle
546513861f Some bug fixes for more complicated types introduced with gparity 2015-08-13 10:50:34 +01:00
Peter Boyle
6ab73c5512 Gparity test added; partial implementation -- this is Chris K's doubled lattice only
and have to regress this with the 2 flavour implementation.
2015-08-12 09:49:33 +01:00
Peter Boyle
8a0be42080 Gparity test added; partial implementation -- this is Chris K's doubled lattice only
and have to regress this with the 2 flavour implementation.
2015-08-12 09:49:33 +01:00
Peter Boyle
9183380946 Gparity test added; partial implementation -- this is Chris K's doubled lattice only
and have to regress this with the 2 flavour implementation.
2015-08-12 09:49:33 +01:00