Dear Authors,

Please find below the comments from the Institutional Review by the DESY group.

We have carefully read your paper HIN-18-018. The paper contains very interesting measurements, some of them performed for the first time using heavy ions, congratulations!
Nevertheless, we also see some needs for improvements in several important areas, as we 
describe in the following. 
For convenience, we have divided our comments, into: General, Physics (type B), and Style (type A).

Thank you very much and good luck with the next steps.

===================================

* General comments
—————————

Given that the target journal is JHEP, the paper would very much benefit from extended 
explanations and plots that could improve the confidence in the correctness of the presented 
results, see comments below.

** It is not entirely clear how antiquark jets are treated in this analysis. In the text (Section 7) only u quarks (with positive charge) and d quarks (with negative charge) seem to be mentioned, but the 
fact that there should be an approximately similar number of ubar and dbar jets with opposite charge 
seems to be ignored. In the caption of Figure 1 "antiquark jets" seem to be considered together
with "heavy jets" (s, c, and b). Is this really the case? Can you elaborate on this?
By the way, declaring s as "heavy" is quite unusual… please consider rephrasing.

** The description of the statistical procedures are not always clear and some of them seem doubtful 
(or maybe even wrong):

— L155-182: The D'Agostini unfolding is applied twice, first correcting for background and in a second step for tracking inefficiencies. However, after the first step the unfolded results are bin-to-bin correlated and cannot be used as bin-to-bin uncorrelated input (as expected by the D'Agostini algorithm) for another D'Agostini unfolding. Note that D'Agostini with early stopping is doing a smoothing, so you effectively smooth the jet charge spectra twice, therefore, probably over-smoothing.

— L176 ff:  The argument on how to chose the D'Agostini iteration number, based on a balance between statistical uncertainties and the bias towards the PYTHIA prediction is hand-waving and needs to be substantiated or replaced with an objective criterion. Please take note that optimising the balance for a point-wise estimation in each bin gives you no clear information about the correlated distortion of the whole spectrum towards the MC prediction. It would be best to make a closure or calibration test by unfolding and fitting PYTHIA MCs generated with varying fractions of gluons (e.g. 0.3, 0.4, 0.5, 0.6), using the nominal PYTHIA for response matrix and bias vector and plotting extracted gluon fractions (from the final template fits) vs generated fractions.

— L178: The biases in the unfolding come from two sources:
  (1) The modelling of the response matrix.
  (2) The starting vector in the D'Agostini unfolding, which is usually the Signal MC spectrum. This vector is sometimes called the bias vector.

In the unfolding tests with 50% variations of quark and gluon fractions, you have taken into account source (1), since you unfold these data using the nominal MC for the response matrix. The question is: what you have done for (2)? Did you also use the nominal MC spectrum for the bias vector or did you use the spectrum after the 50% variation? In the latter case, this is not a complete bias test. The 50% variation size seems a bit arbitrary. In any case, some closure test as indicated above (see the point to L176) would give the most clear idea on biases.

— L198-199 and L221: you are talking about adding uncertainties in quadrature, but it is not clear to what this is exactly applied: to the bin-wise unfolded jet-charge measurements or to the final gluon fraction fit results? If it is applied per bin of jet-charge it would be wrong, since the bin-to-bin correlations are not taken into account.

— Section 7, Template fitting: please elaborate how the chi2 is constructed for the template fit. Is this done with the most simple form of a chi2 that is a sum over the squared pulls between fit and data? Or is this done using vectors of residuals between data and fit, lets call them D, and taking 
Chi2 = D^t V^-1 D, where V is the covariance matrix of the data from unfolding? The latter way is the correct way, while the other way is wrong.

** Lack of information and supporting material in the paper:

— Throughout the text, is not always clear which selections/methods are applied to pp and PbPb. This needs to be described more clearly.

— Section 6: a complex measurement is performed and one would expect to see some control plots at the detector level, before background corrections and unfolding. In particular, one would like to see how well the sum of MC distributions model the data distributions of jet pt and eta, jet multiplicity and jet charge.

— Section 6: There is practically zero information given on the size of background contributions and
on how large smearing effects are in the unfolding procedure. Looking into the AN, it seems that the bin-sizes for the jet-charges are smaller than the detector resolutions; this means that you need a rather strong regularization in the unfolding. A detector level control plot of the jet-charge distribution could be helpful to see how much one needs to correct these spectra to arrive at the unfolded spectra in Fig. 1.

— Section 9 and Fig. 1: u, d and g fractions are extracted, but no information is given on the separate results for u and d and the degree of fit-correlation between them and with g. There is also no information given on the chi2_min value of the fit and NDF. This could provide nice information on the Goodness-of-Fit. In Figure 1 one can see some fit to data discrepancies at the edges of the distributions, are these significant? No discussion is given in the text.

— It is not clear why the figures in the Appendix are not part of the main body of the paper and discussed accordingly. Suggest to include them.

** Interpretation of the results: 

— The analysis in pp and PbPb is not exactly identical (eg selection of the charged tracks, background). There is no explicit comparison between the pp and PbPb results (only results put next to each other, leaving the comparison to the reader), but how sure are we that we are measuring the same in pp and PbPb, given the differences in the analyses? This should be discussed in the paper, even if briefly.

— The discussion of the results is rather scant. The conclusions and physics interpretations are based on optical inspection of the Figs. 2-4. Ideally one would have done chi2 tests, for instance based on the difference of the gluon fractions observed in pp and PbPb data (for given centrality, where one takes the squared difference and divides this by the squared error for the difference, using error propagation). Also, it seems that we are not confirming the results reported in Refs [11, 12] — this should be discussed in Section 9 as well.

 

* Physics (TYPE B) comments:
——————————————

- General: The title talks about "quark-like" and "gluon-like" jet fractions, while the introduction (L13, 15, 29, 42) and most of the paper talk about “quark" and "gluon" jet fractions. Either, change the title, or introduce the terms "quark-like" and "gluon-like" somewhere in the Introduction and motivate
why this nomenclature is important.

- Title: "study" -> "measurement"

- Abstract: "is proposed to estimate" -> "is used to measure"

- L11-12: "... comparing PbPb and pp data as a function of collision centrality." ==> compared is pp data with PbPb data in different collision centrality regions. Please rephrase.

- L24: “width” —> “variance” ? Also you are referring to the “distribution” of the jet charge, or? Please add it. 

- L31: this is the first time the jet charge is measured in PbPb, should be mentioned already here.

- L32: why is 2017 data at 5.02 TeV not used?

- L34-37: the description and definition of "background" is rather unclear. It might help to state explicitly that "signal" are single jets within events (and not full events), and "background" refers to other partons in the same events whose shower is partly reconstructed as part of the signal jet (is that right?). You are talking about something that is measured at least 1.5. units away in eta, but it is not clear how that effects the energy deposits and tracks in the cone around the jet. Please clarify.

- L38: “unfolded for detector and background effects” —> “corrected for detector and background effects using an unfolding procedure” or similar 

- L38: mention that you are performing the unfolding to particle level and define it (see also comment to L156).

- L45: Kappa is not a weighting factor and is undefined here. Please define.

- L54: Please motivate the choice "abs(eta)<1.6". None of the following detector coverage details explains this choice. Also, in L119 a cut of "abs(eta)<1.5" is applied, also without motivation. Why these many different numbers? Please explain in the paper.

- L75: please define v_z 

- L78 and everywhere else HF is mentioned: please be consistent throughout the text in the way you refer to HF: calorimeter, detector, region. Also, would it be relevant to mention here the range in eta?

- L82: Why is Pythia6 used in the analysis and not Pyhia8? Also, use consistent naming convention in text and figures. Suggest to define here “Phythia6” and use it hereafter in the text, as done in many other CMS papers.

- L90-95: How well is the centrality measured? Are there strong migration effects when comparing generator to detector level centrality and one should apply an unfolding procedure? If not, then one would like to know at least how pure the bins are (roughly). 

- L90: what is the relevance of having the eta region mentioned here? Shouldn’t it appear earlier in the text?

- L94: How are the centrality intervals chosen? The quantitative criterion should be explained in the text.

- L96: this sentence is hard to read, why not reverse the logics:
Central collision events have more binary nucleon-nucleon interactions
==> thus there are on average more central jets produced than in non-central collisions 
==> thus requiring the presence of a jets biases the data sample towards more central collisions
Suggest to find a better word than “binary”, if possible.

- Par L96-102: The details of the centrality-based reweighting remain unclear/obscure. Is this done in a continuous way or in a discreet way, using the centrality bins of the measurements? Is there not
a systematic uncertainty of the reweighting? Can you provide a reference for this reweighting?

- L102: Please add a comment about PU, even if it is to say that it is negligible in pp.

-L104-137: It is not entirely clear which parts of the jet and track reconstruction are common for pp and PbPb collisions, and which are unique. The current structure of the description is convoluted. We would recommend describing first what is common, or a procedure for one type of collisions, and then summarize what is different for the other type of collisions.

- L113-119: Is this applied to both pp and PbPb? How is the matching between calo jets and tracks done? How sure are we that the correction is not biasing the result? More information is needed here. Also, can you provide a reference for this correction?

- L120: I guess you mean "25% of pp events with one selected jet also contain another selected jet", not "25% of all pp events contain two jets". Please reformulate in an unambiguous way. Also, what about PbPb and multiple jets there? 

- L127: Statistically, the p-value for the chi2 with the given NDF is a good quantity for a Goodness-Of-Fit test, so why dividing the chi2 in addition by the number of tracker layers hit? Where does the 0.15 criterion come from? Can you provide a reference?

- L135: You could mention that the tracking efficiency numbers here and later are estimated from the MC simulation.

- Equation 1: suggest to remove the brackets and find a less confusing way to express the product of q_i and p_T^i. Maybe you can try with p_{T, i}^k ? Also, please write “jet” in the same font.

- L146: is there a PV requirement on the tracks associated to the same jet?

- L149: Here you speak about "combinatorial and long-range correlated background", and in L35 you are talking about "uncorrelated and long-range correlated contributions”. This points, again, towards an unclear definition of background.

- Section 6, title and subsequent text: the phrase ”Unfolding of background" is very confusing. In general, we usually talk about correcting for background effects or subtracting it, and then doing the unfolding for detector resolutions and inefficiencies. Please rephrase.

- It is not fully clear how the unfolding is done for pp.

- It is not clear what the background for pp is. Also, how large is it?

- L156: Be more specific on what is the "final-state particle level", e.g. is it before or after hadronization? Are there any cuts applied on generator level?

- L162: “derived from PYTHIA” —> “derived from the PYTHIA simulation sample” or similar

- L163: The regularization used in the unfolding needs to be mentioned somewhere in this section, maybe already here.

- L167:  "These values are mapped.." In the sentence before you talk about the response matrices and it reads is as if the response matrix values are mapped; this is not correct. One maps the bin-wise counts in the data at detector level with background to the detector level without background, using the inverse of the response matrix. Please rephrase.

- L168: “unfolding matrices” vs “response matrices” used elsewhere ==> please use a consistent naming convention throughout all the paper

- L170: "using jet-track pairs from" —> please remove

- L170-175: You write that you use the jet-charge measured in the mixed events. Why is it not sufficient to directly measure the background contributions in the minimum-bias event and compare that to HYDJET background contribution? Do we really need to mix it first with a good event?
The good event already contains some background itself, can this be neglected?

- L174: “unfolding for background effects” —> “subtracting the background from the signal”, or something similar.

- L178: Why 50%? What happens for 30% or 70%? Please justify the number and/or give a reference.

-L179-182: Please specify the exact number of iterations used in each case.

- L180 : "is optimal in balancing the bias towards the MC distribution" ? This sentence is not clear, please rephrase. Also, how does it affect the fractions of quark and gluon jets?

- L187: “e” should be defined (and written in roman font).

- L195 : why fix relative fractions of heavy quark jets ? need to reduce ndf in fit ?
- L196-199 : treatment of uncertainties already explained in Section 8; suggest to move any additional information to the corresponding lines in Section 8.

- L197: by how much are the HF fractions for the quarks changed? What about the antiquarks?

- L203 : "and the relative fractions of quarks and gluons for such jets are assigned directly from MC" => is this justified ? Maybe good to refer to Section 8 where this syst is discussed ?
- L208: "... PbPb data than PYTHIA ... " —> "... PbPb data than it is modeled in PYTHIA ..."?

- L216 : “extra correction factor” obtained how? This needs explanation/justification, or at the very least, a reference.

- Section 8: it is unclear how all the systematic uncertainties are evaluated, as information is provided only for some of them. Is the unfolding and fitting redone for all sources for up and down variations and then compared to the nominal case? This needs to be clearly described 

- L224: Why is there a difference for positive and negative particles? Is this 0.5% for data or for MC? Both for pp and PbPb? Please clarify.

- L230: Can you give references for these numbers? Where do they come from?

- L243 : what do the 4-7% refer to? different bins? Same comment to L251. Please justify and add a reference if available.
- L245: ".. the uncertainty propagated via bin-to-bin correlations as a result of the regularization process." ==> sorry, the meaning of this sentence is unclear. Please claify/rephrase.

- L246: This is the first time when the regularization is mentioned in the paper. Regularization has to be described in the unfolding chapter, as well as the way to determine its strength. See comment above.

- L253-258: this is only for PbPb, what are you assuming for pp?

- L255: what you mean exactly by "signal event" here, is this data or MC?

- L265-L273: the discussion of these underlying events is a bit abstract, perhaps one could add 
some more information about how much particle multiplicities could change (e.g. in percent) due to this systematic? 

- L265: “sit on” —> please rephrase to something less slang 

- L266: “jet [pT] spectrum”? Since this is a steeply falling distribution, it would be good to show a data-MC comparison plot to see the agreement in the description of the data by the simulation
- L270-273: “To calculate this difference …” ==> this procedure needs to be better explained/justified. At the very least, a reference (if available) should be added.

- L271: should it be “R” or “delta_r”? Please check.

- L279-280: Suggest to drop the bin-size variations from the systematics; you just sample statistical fluctuations here that appear within a bin that is split into two bins, and these could easily lead to overcovering (too large) uncertainties.

- L282: what exactly is varied, and by how much? This is unclear, please explain.
- L283: “A summary ..” should be a different paragraph.

- L284, Table1: Can you add the statistical uncertainty to the table? Also, what does the dash line for “background modelling and fluctuation mean”? That is it not considered? That it is negligible? Please explain both in the text and in the caption of the table.

- L285: Why are there no systematic uncertainties for JES, parton shower or underlying event tune? I could imagine there are non-negligible contributions from these effects. Have you also checked differences wrt Herwig7 or Sherpa? 

- L292, Figure 1 (and 5, 6):
— treatment of the up and down antiquark jets is unclear (see general comment). Please elaborate.
— please add “Centrality” explicitly in the plots next to the intervals, and also in the caption
— please show the total uncertainty in data, separating in statistical and systematic, and not just the systematic uncertainty
— Please comment in the text about the data-MC discrepancies observed at jet charge values ~ -1. Should we be concerned? 

- L293: imprecise: " to be relatively flat " —> better, something like: "with no significant deviation from one"

- L296, and Figure 2: There is no discussion on the systematics of the standard deviations of the jet charge distributions. Are they actually included? If not, they should be. Btw: can the jet charge be larger than unity? If yes, then they are chopped in Figure 2. The tails outside unity might have an effect on the standard deviation. 

- Figure 3, 4: 
— How is "gluon-like jet fraction" defined? Especially, does it mean gluon vs gluon+up+down, or gluon vs gluon+up+down+other? Please define in the text
— Please show the total (stat and syst) uncertainty.
— It would be nice to have plots for the other kappa values.

- L314 ff and Figure 4:  One can see some deviations from the 0.6 horizontal line, e.g. in the third panel at kappa = 0.7. Is this significant? It should be mentioned in the text.

- L318: Please comment on how the results in this analysis compare to those in Refs[10, 48]. 

- L320-322: This statement needs a reference, at the very least.

- L323 : “no significant modification” ==> in contradiction with previous CMS observation of modification at low pt... (supporting jet quenching models)? Is there a possible/preferred explanation? Please elaborate more in this. 

- L332-335: One can understand that conclusions based on PbPb measurements, i.e. results are independent w.r.t. collision centrality, are also true for pp collisions at different centralities. However, in this paper, technically, only *one* centrality was measured for pp collisions. One could simply state conclusions for PbPb separately and only then specify that distributions from pp measurements follow similar trends as the ones observed from the PbPb measurements, as well as specifying what could this mean physics-wise (e.g. similar to what is done in L321-323).

- L334: how is the centrality defined for pp?

* Style (TYPE A) comments:
—————————————

- Throughout all the text: “cuts” is jargon —> “requirements”, “thresholds”

- Intro : PbPb and pp in words ?

- L3: suffer -> undergo ?

- L9: “jets [4-8][,] as well as” —> comma missing in front of first “as”

- L14: “[the] quenching”?

- L25: "... on both[,] jet energy and [jet] size[,] can ..." —> commas missing

- L33: reverts pp and Pb to be consistent with L31 ? 

- L35 : “is defined in terms of...” ? 
- L38 : “are unfolded to account for” ? 
- L39,94 : “overlap between the” ? 
- L52-55 : need to spell out eta and phi ? ("pseudorapidity", “azimuthal”) 
- L55 : “region of” -> “region corresponding to” ? 
- L57 : “nominal IP” ? 
- L64 : “which is used” ? 
- L68 : “a distance parameter of” 
- L74 : “present” -> “reconstructed” ?

- L93 : “given by” -> “corresponding to”?

- L97: "... requiring a jet to be present in an event[,] biases the data sample ..." —> comma missing
 
- L108 : “underlying events”?
 - L121 : “data and simulations” -> “and simulated events” ? 
- L127 : “tracks must have at least 11 hits" -> “tracks must be reconstructed from/associated with at least...”? 
- L129 : “a selection requirement of less than 3 sd” -> “for both collision systems, it is required that the significance... be less than 3 sd..., in order to decrease...” ? 
- L142 : remove “magnitude” or mention it also for pTjet ? idem “relative to beam axis” ? 
- L147,206 : dr -> dR (consistency) 
- L149 : “well resolved” -> “well simulated” ? 
- L150 : “Theoretical predictions suggest that [a parameter value] κ 0.5 is the most sensitive” ? 
- L152 : “to maintain” -> “, in order to retain” ?

- L153: "... to both[,] hard and soft ..."

- L155 : “to compare with” -> “to allow for a comparison with” ? 
- L161 : “iteration method” -> “iterative method” ? 
- L162 : “for pp [collisions]” ? 
- L178 : “obtained with PYTHIA” ? 
- L180 : "is optimal in balancing the bias towards the MC distribution" ? Please rephrase
 - L185 : “in fitting” -> “to fit” ?
 - L186 : rephrase ("Measurements... make up the dominant proportions") 
 - L194 : missing space “+hydjet”
 - L195 : at least rephrase “by varying the heavy quark jets” (-> “by varying the total fraction of...”?) 
 - L208 : “than in simulations obtained with pythia...” ? ; “increases with increasing centrality of the collisions"?

- L228 : “cited” -> “quoted” ?

- L224: "The difference in [the] tracking …”, “… positive and negative [charged] particles ..."

- L230 : “evaluated” -> “considered” ? rephrase “in pp and ... in pbpb” 
- L232 : “fashion similar” -> “similar fashion”? 
- L233: "... positive and negative [charged] ..."

- L249 : which “quantities”? rephrase?
 - L253-258 : already explained ? redundant ? 
- L270 : rephrase ; “distributions of the particle multiplicities within cones, chosen...” ? 
- L288 : “sample bin”? 
- L314 : “consolidated”? 
- Legend Fig.3 : “shown by” -> “represented by”? 

- L334, and throughout text : replace “measurements from Pythia/...” by “simulations…”

- Figures:
— please use consistent naming conventions for all figures (e.g, “Q”, vs “jet charge”)
— “Pythia” —> “Pythia6” (see previous comment)
— please show always total uncertainties, possibly separating in stat and syst, and do it consistently for all figures
— “cut” —> “threshold”
— Fig 2: crosses for Pythia6 could be replaced with other markers (lines) to improve visibility.