SMP-18-001 Measurement of electroweak WZ production Comments on CWR draft by O. Behnke: General comments: ----------------- - In several places you report the measurement of the EW WZ significance, e.g. Table 2 and l.321 and l.355/356. It would be better to report the measurement of the electroweak WZ production cross section, because this is the physics observable of interest and not the significance. - On the maximum likelhood fits and limit settings there are some questions: - Do you use the Higgs combine tool? - Do you use in general a likelihood based on Poisson statistics or gaussian approximation? - Do you take the MC stat. uncertainties of MC templates into account with the Barlow Beeston method? Do you use gaussian approximation for these unc.? - Do you apply template morphing for nuisance parameters? (use +- 1 sigma variations and nominal template for interpolation and extrapolation of dependence of template on nuisance parameter, as done in combine tool) - What PDFs do you use for nuisance parameters such as JES: e.g. log-normal type? - you seem to use the asymptotic (means large stat.) formulae from the Cowan paper [56]. Are you sure these asymptotic formulae are valid? For some channels the stat. in some bins look close to zero events. Within the combine tool one can make cross checks using toy experiments. - For the extraction of the EW ZW cross section, why don't we also provide the information on the extracted QCD ZW cross section that seems to be fitted also? TYPE B: ------- Title: --> 'Measurement of WZ production ..' Reason: it seems that the WZjj measurement is the main result of this paper and not the electroweak WZ production Abstract: Suggest to have first the sentence "The total WZ plus two jet production cross ssection is measured in fiducial regions with enhanced contributions from.." and then "The cross section contribution from electroweak WZ production within this region is measured with an observed..." Fig. 1, is the diagram (a) really correct, isn't one of the q quarks ending as q and not q'? l.28 'These kinematic selections are used to distinguish the EW.. from ..QCD-induced process, which is considered as backfground' --> is this sentence really needed and does it make sense, for the total WZjj measurement QCD-induced process is signal.. also it is not correct to say that these kin. selections are used to distinguish, because you separate the contributions within (and not with) the given kin. selection from distributions of observables. l.54-60 the detailed discussion of the muon system vs. eta and track pt resolutions.. where is this later relevant? l.81 'resonant W boson propagator', perhaps a stupid question, but could there be also a 'resonant Z boson propagator' l.91 why does one need to merge different jet multiplicities? l.96, how can the diagrams in Fig.1 (a) and (b) interfere, they have a different final state. l.106 why use four-flavour scheme and scale to five-flavour scheme, why not using five-flavour scheme from the start? l.121 --> 'For all simulations used in this analysis the NNPDF3.0 [36] set ...' Reason: Otherwise it sounds as if you are just referring to the sim. mentioned on top of p.4 l.136-139 --> it is clear that charged tracks can be associated to a primary vertex candidate, but how can the associated pt_miss (as mentioned) and how can a jet be associated to a primary vertex? In particular the pt_miss I find confusing. l.180-188, is the rather detailed listing of eff. vs eta so crucial? l.204 b-tagging works until |eta|<2.4, so what is done for 2.4 'primarily ttbar and Z+jets' par l.233-240 for the measurement of WZjj the QCD WZ production is signal. Here you treat is as background and do a normalisation using control regions. Is this consistent with the later extraction of WZjj and electroweak WZ cross section? l.241 in general when you use 'loose' selections as control regions which assumptions do you make on the composition of the event in these regions? e.g. neglect signal components? Are the events in the loose region statistically independent from the tight regions? l.245 '"Tight-to-loose" transfer factors' isn't that '"Loose-to-tight'" transfer factors par l.247-256 got completely lost here, find the description not understandable. you introduce many different samples: DY and ttbar, dijet events (l.251, how are they defined?) and Z+jet events.. in particular it is confusing that you start in l.247 with 'This method is validated' and then in l.250' These results are cross-checked' I did not understand what was really checked with what and what you mean by 'These results' par l.257-263 again got lost here. What do you mean by 'fakeable object control regions'?? l.259 'nonprompt background, ..with the normalisation per channel taken from the ratios measured in WZ plus two jet events.' Isn't that interfering with your signal extraction for WZjj?? Is there a short-circuit here in the method? l.266 start a new par at "The JES and JER ..." otherwise it looks like that this and the following sentences in the same paragraph should cover also the modeling uncertainties. l.268 'acceptance in', there is not 'acceptance into' l.269 2D discriminant was not yet introduced. l.270 'The modelling uncertainty' l.270 'on the QCD WZ process' ??? it is not always background! par l.270-282 it is really hard to follow the description of the model uncertainties, what enters finally as uncertainties on shapes and on normalisations. Please try to improve clarity. par l.283-290, also find this paragraph hard to understand/follow. l.292 'tens of percent' you mean like 10, 20 30 percent? par l.295-303 could there be some double counting of statistical uncertainties also counted as systematical uncertainties? l.308 what is a 'fit distribution'? l.309 'bin-by-bin' Why treating each bin as uncorrelated from the other here? Isn't that a fully bin-to-bin correlated sys. source? l.310 is the uncertainty from the MC replica sets really a Gaussian uncertainty? how is the width of the gaussian defined? Table 2 --> 'freezing the set of associated nusiance parameters' l.325 you fit to the observed event yields, right? So take out 'and expected' If the expected events yield have an uncertainty than one is fitting the expected event yields to control data or to MC data (Barlow Beeston method). l.329 what are 'scaling nuisance parameters' this term seem to be not well defined. l.325 a general question on the Max. lh fit. Do you use Poisson statistics, Fig. 2 the hatched band, labeled "Stat. Unc." is not explained in the caption Fig. 2 'The solid symbols' not sure if I understand what you mean here. Fig. 2 last line, the normalisations are not shown. --> 'The predicted yields are shown with their pre-fit normalisations.' or similar Fig. 3 'The predictions are shown with using the Post-Fit values for their normalisations.' l.365 ff. 'The significance of the excess is quantified by calculating the local p-value for an upward fluctuation of the background using a profile likelihood ratio test statistic and asymptotic formulae [56]. This has nothing to do with CLs (references). l.366 what you mean with stat. uncertainties, the ones of the data or also related to control regions or MC samples, etc.? Equation (3): this is the fit result, is the uncertainty determined from the variation of -2ln L by +1 from the min. value.. where L is the profiled likelihood --> better state that. Do you have more information on the best fit result, e.g. how much are nusiance pars shifted? You could also do a GOF-test, see https://twiki.cern.ch/twiki/bin/view/CMS/StatComGOF using the Baker-Cousins advocated likelihood chi2 test. l.390 'using the CLs criterion' it is not a method. Please take note tha [59] = [54] Fig. 4 'shown as a filled histogram.' Aren't all backgrounds shown as filled histos? Table 4: when you give 2-sided confidence intervals, I would preferto call them confidence intervals and not limits. Are you sure these are 95% C.L. and not 90% C.L. intervals? Fig. 5 how are the contour lines exactly defined? Are you sure you take the 2d effect into account? (e.g. a Delta_chi2 = +1 means 68% C.L. for 1D but only ~40% for 2D) l.410 'CLs criterion' TYPE A ------- l.191 'single lepton trigger' l.209 'negligible effect' l.383 'formulation' Fig. 6, top,bottom --> left, right