Skip to main content
Advertisement
  • Loading metrics

Viral dynamics of acute SARS-CoV-2 infection and applications to diagnostic and public health strategies

  • Stephen M. Kissler ,

    Contributed equally to this work with: Stephen M. Kissler, Joseph R. Fauver, Christina Mack

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Writing – original draft, Writing – review & editing

    Affiliation Department of Immunology and Infectious Diseases, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States of America

  • Joseph R. Fauver ,

    Contributed equally to this work with: Stephen M. Kissler, Joseph R. Fauver, Christina Mack

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, Connecticut, United States of America

  • Christina Mack ,

    Contributed equally to this work with: Stephen M. Kissler, Joseph R. Fauver, Christina Mack

    Roles Conceptualization, Data curation, Investigation, Writing – original draft, Writing – review & editing

    Affiliations Real World Solutions, IQVIA, Durham, North Carolina, United States of America, Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States of America

  • Scott W. Olesen,

    Roles Conceptualization, Formal analysis, Methodology, Software, Visualization

    Affiliation Department of Immunology and Infectious Diseases, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States of America

  • Caroline Tai,

    Roles Conceptualization, Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing

    Affiliation Real World Solutions, IQVIA, Durham, North Carolina, United States of America

  • Kristin Y. Shiue,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliations Real World Solutions, IQVIA, Durham, North Carolina, United States of America, Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States of America

  • Chaney C. Kalinich,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliation Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, Connecticut, United States of America

  • Sarah Jednak,

    Roles Data curation, Investigation, Writing – review & editing

    Affiliation Department of Health Management and Policy, University of Michigan School of Public Health, Ann Arbor, Michigan, United States of America

  • Isabel M. Ott,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, Connecticut, United States of America

  • Chantal B. F. Vogels,

    Roles Methodology, Writing – review & editing

    Affiliation Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, Connecticut, United States of America

  • Jay Wohlgemuth,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Quest Diagnostics, San Juan Capistrano, California, United States of America

  • James Weisberger,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Bioreference Laboratories, Elmwood Park, New Jersey, United States of America

  • John DiFiori,

    Roles Conceptualization, Data curation, Methodology, Writing – original draft, Writing – review & editing

    Affiliations Hospital for Special Surgery, New York, New York, United States of America, National Basketball Association, New York, New York, United States of America

  • Deverick J. Anderson,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Duke Center for Antimicrobial Stewardship and Infection Prevention, Durham, North Carolina, United States of America

  • Jimmie Mancell,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Department of Medicine, University of Tennessee Health Science Center, Memphis, Tennessee, United States of America

  • David D. Ho,

    Roles Data curation, Methodology, Writing – review & editing

    Affiliation Aaron Diamond AIDS Research Center, Columbia University Vagelos College of Physicians and Surgeons, New York, New York, United States of America

  • Nathan D. Grubaugh ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

    ‡ These authors are joint senior authors on this work.

    Affiliation Department of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, Connecticut, United States of America

  •  [ ... ],
  • Yonatan H. Grad

    Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

    ygrad@hsph.harvard.edu

    ‡ These authors are joint senior authors on this work.

    Affiliation Department of Immunology and Infectious Diseases, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, United States of America

  • [ view all ]
  • [ view less ]

Abstract

SARS-CoV-2 infections are characterized by viral proliferation and clearance phases and can be followed by low-level persistent viral RNA shedding. The dynamics of viral RNA concentration, particularly in the early stages of infection, can inform clinical measures and interventions such as test-based screening. We used prospective longitudinal quantitative reverse transcription PCR testing to measure the viral RNA trajectories for 68 individuals during the resumption of the 2019–2020 National Basketball Association season. For 46 individuals with acute infections, we inferred the peak viral concentration and the duration of the viral proliferation and clearance phases. According to our mathematical model, we found that viral RNA concentrations peaked an average of 3.3 days (95% credible interval [CI] 2.5, 4.2) after first possible detectability at a cycle threshold value of 22.3 (95% CI 20.5, 23.9). The viral clearance phase lasted longer for symptomatic individuals (10.9 days [95% CI 7.9, 14.4]) than for asymptomatic individuals (7.8 days [95% CI 6.1, 9.7]). A second test within 2 days after an initial positive PCR test substantially improves certainty about a patient’s infection stage. The effective sensitivity of a test intended to identify infectious individuals declines substantially with test turnaround time. These findings indicate that SARS-CoV-2 viral concentrations peak rapidly regardless of symptoms. Sequential tests can help reveal a patient’s progress through infection stages. Frequent, rapid-turnaround testing is needed to effectively screen individuals before they become infectious.

Introduction

A critical strategy to curb the spread of SARS-CoV-2 is to rapidly identify and isolate infectious individuals. Because symptoms are an unreliable indicator of infectiousness and infections are frequently asymptomatic [1], testing is key to determining whether a person is infected and may be contagious. Real-time quantitative reverse transcription polymerase chain reaction (RT-qPCR) tests are the gold standard for detecting SARS-CoV-2 infection. Normally, these tests yield a binary positive/negative diagnosis based on detection of viral RNA. However, they can also quantify the viral titer via the cycle threshold (Ct). The Ct is the number of thermal cycles needed to amplify sampled viral RNA to a detectable level: the higher the sampled viral RNA concentration, the lower the Ct. This inverse correlation between Ct and viral concentration makes RT-qPCR tests far more valuable than a binary diagnostic, as they can be used to reveal a person’s progress through key stages of infection [2], with the potential to assist clinical and public health decision-making. However, the dynamics of the Ct during the earliest stages of infection, when contagiousness is rapidly increasing, have been unclear, because diagnostic testing is usually performed after the onset of symptoms, when viral RNA concentration has peaked and already begun to decline, and is performed only once [3,4]. Without a clear picture of the course of SARS-CoV-2 viral concentrations across the full duration of acute infection, it has been impossible to specify key elements of testing algorithms such as the frequency of rapid at-home testing [5] that would be needed to reliably screen infectious individuals before they transmit infection. Here, we fill this gap by analyzing the prospective longitudinal SARS-CoV-2 RT-qPCR testing performed for players, staff, and vendors during the resumption of the 2019–2020 National Basketball Association (NBA) season.

Methods

Data collection

The study period began in teams’ local cities from June 23 through July 9, 2020, and testing continued for all teams as they transitioned to Orlando, Florida, through September 7, 2020. A total of 68 individuals (90% male) were tested at least 5 times during the study period and recorded at least 1 positive test with Ct value < 40. Most consecutive tests (85%) were recorded within 1 day of each other, and fewer than 3% of the intervals between consecutive tests exceeded 4 days (S1 Fig). Many individuals were being tested daily as part of Orlando campus monitoring. Due to a lack of new infections among players and team staff after clearing quarantine in Orlando, all players and team staff included in the results predate the Orlando phase of the restarted season. A diagnosis of “acute” or “persistent” infection was abstracted from physician records. “Acute” denoted a likely new infection. “Persistent” indicated the presence of virus in a clinically recovered individual, likely due to infection that developed prior to the onset of the study. There were 46 acute infections; the remaining 22 individuals were assumed to be persistently shedding SARS-CoV-2 RNA due to a known infection that occurred prior to the study period [6]. This persistent RNA shedding can last for weeks after an acute infection and likely represents noninfectious viral RNA [7]. Of the individuals included in the study, 27 of the 46 with acute infections and 40 of the 68 overall were staff and vendors. The Ct values for all tests for the 68 individuals included in the analysis, with their designations of acute or persistent infection, are depicted in S2S5 Figs. A schematic diagram of the data collection and analysis pipeline is given in Fig 1.

thumbnail
Fig 1. Illustration of the analysis pipeline.

Combined anterior nares and oropharyngeal swabs were tested using a RT-qPCR assay to generate longitudinal Ct values (“Raw data”; red points) for each person. Using a statistical model (see S6 Fig for a schematic of the model), we estimated Ct trajectories consistent with the data, represented by the thin lines under the “Model fits” heading. These produced posterior probability distributions for the peak Ct value, the duration of the proliferation phase (first potential detectability of infection to peak Ct), and the duration of the clearance phase (peak Ct to resolution of acute infection) for each person. We estimated population means for these quantities (under the heading “Population estimates”). The model fits also allowed us to determine how frequently a given Ct value or pair of Ct values within a 5-unit window (blue bars, under the heading “Predicting infection from Ct”) was associated with the proliferation phase, the clearance phase, or a persistent infection. Finally, the model fits allowed us to measure the “effective sensitivity” of a test for predicting future infectiousness. The schematic illustration titled “Measuring effective sensitivity” depicts the relationship between testing lags and the ability to detect infectious individuals at a gathering. The illustrated viral trajectory surpasses the infectiousness threshold (dotted line) at the time of the gathering (vertical grey bar), so unless this individual is screened by a pre-gathering test, he or she would attend the event while infectious. One day prior to the gathering, the individual’s infection could be detected by either a rapid test or a PCR test. Two days prior to the event, the individual’s infection could be detected by a PCR test but not by a rapid test. Three days prior to the event, neither test would detect the individual’s infection. Ct, cycle threshold; LOD, limit of detection; RT-qPCR, quantitative reverse transcription polymerase chain reaction.

https://doi.org/10.1371/journal.pbio.3001333.g001

Ethics

Residual de-identified viral transport media from anterior nares and oropharyngeal swabs collected from NBA players, staff, and vendors were obtained from Quest Diagnostics or BioReference Laboratories. In accordance with the guidelines of the Yale Human Investigation Committee, this work with de-identified samples was approved for research not involving human subjects by the Yale Institutional Review Board (HIC protocol #2000028599). This project was designated exempt by the Harvard Institutional Review Board (IRB20-1407).

Statistical analysis

Due to imperfect sampling, persistent viral shedding, and test uncertainty near the limit of detection, a straightforward analysis of the data would be insufficient to reveal the duration and peak magnitude of the viral trajectory. Imperfect sampling would bias estimates of the peak viral concentration towards lower concentrations/higher Ct values since the moment of peak viral concentration is unlikely to be captured. Persistent shedding and test uncertainty would bias estimates of the trajectory duration towards longer durations of infection. To address these problems, we used a Bayesian statistical model to infer the peak Ct value and the durations of the proliferation and clearance stages for the 46 acute infections (Fig 1; S1 Text). We assumed that the viral concentration trajectories consisted of a proliferation phase, with exponential growth in viral RNA concentration, followed by a clearance phase, characterized by exponential decay in viral RNA concentration [8]. Since Ct values are roughly proportional to the negative logarithm of viral concentration [2], this corresponds to a linear decrease in Ct followed by a linear increase. We therefore constructed a piecewise linear regression model to estimate the peak Ct value, the time from infection onset to peak (i.e., the duration of the proliferation stage), and the time from peak to infection resolution (i.e., the duration of the clearance stage). This allowed us to separate the viral trajectories into the 3 distinct phases: proliferation (from the onset of detectability to the peak viral concentration, or to to tp in S6 Fig), clearance (from the peak viral concentration to the resolution of acute infection, or tp to tr in S6 Fig), and persistence (lasting indefinitely after the resolution of acute infection, or after tr in S6 Fig; see also Fig 1). Note that for the 46 individuals with acute infections, the persistence phase is identified using the viral trajectory model, whereas for the 22 other infections, the entire series of observations was classified as “persistent” due to clinical evidence of a probable infection prior to the start of the study period. We estimated the parameters of the regression model by fitting to the available data using a Hamiltonian Monte Carlo algorithm [9] yielding simulated draws from the Bayesian posterior distribution for each parameter. Full details on the fitting procedure are given in S1 Text. Code is available at https://github.com/gradlab/CtTrajectories [10].

Inferring stage of infection

Next, we determined whether individual or paired Ct values can reveal whether an individual is in the proliferation, clearance, or persistent stage of infection. To assess the predictive value of a single Ct value, we extracted all observed Ct values within a 5-unit window (e.g., between 30.0 and 34.9 Ct) and measured how frequently these values sat within the proliferation stage, the clearance stage, or the persistent stage. We measured these frequencies across 10,000 posterior parameter draws to account for the fact that Ct values near stage transitions (e.g., near the end of the clearance stage) could be assigned to different infection stages depending on the parameter values (see Fig 1, bottom right). We did this for 23 windows with midpoint spanning from Ct = 37.5 to Ct = 15.5 in increments of 1 Ct.

To calculate the probability that a Ct value sitting within a 5-unit window corresponded to an acute infection (i.e., either the proliferation or the clearance stage), we summed the proliferation and clearance frequencies for all samples within that window and divided by the total number of samples in the window. We similarly calculated the probability that a Ct sitting within the 5-unit window corresponded to just the proliferation phase.

To assess the information gained by conducting a second test within 2 days of an initial positive test, we restricted our attention to all samples that had a subsequent sample taken within 2 days. We repeated the above calculations for (a) consecutive tests with decreasing Ct and (b) consecutive tests with increasing Ct. That is, we measured the frequency with which a given Ct value sitting within a 5-unit window, followed by a second test with either a lower or a higher Ct, sat within with the proliferation, clearance, or persistence stages.

Measuring the effective sensitivity of screening tests

The sensitivity of a test is defined as the probability that the test correctly identifies an individual who is positive for some criterion of interest. For clinical diagnostic SARS-CoV-2 tests, the criterion of interest is current infection with SARS-CoV-2. Alternatively, a common goal is to predict infectiousness at some point in the future, as in the context of test-based screening prior to a social gathering. The “effective sensitivity” of a test in this context (i.e., its ability to predict future infectiousness) may differ substantially from its clinical sensitivity (i.e., its ability to detect current infection). A test’s effective sensitivity depends on its inherent characteristics, such as its limit of detection and sampling error rate, as well as the viral dynamics of infected individuals.

To illustrate this, we estimated the effective sensitivity of (a) a test with a limit of detection of 40 Ct and a 1% sampling error probability (akin to RT-qPCR) and (b) a test with a limit of detection of 35 Ct and a 5% sampling error probability (akin to some rapid antigen tests). We measured the frequency with which such tests would successfully identify an individual who would be infectious at the time of a gathering when the test was administered between 0 and 3 days prior to the gathering, given viral trajectories informed by the longitudinal testing data (see schematic in Fig 1). To accomplish this, we drew 1,000 individual-level viral concentration trajectories from the fitted model, restricting to trajectories with peak viral concentration above a given infectiousness threshold (any samples with peak viral concentration below the infectiousness threshold would never be infectious and so would not factor into the sensitivity calculation). For the main analysis, we assumed that the infectiousness threshold was at 30 Ct [11]. In a supplemental analysis, we also assessed infectiousness thresholds of 35 and 20 Ct. We drew onset-of-detectability times (i.e., the onset of the proliferation stage) according to a random uniform distribution so that each person would have a Ct value exceeding the infectiousness threshold at the time of the gathering. Then, we calculated the fraction of trajectories that would be successfully identified using a test with (a) a limit of detection of 40 Ct and (b) a limit of detection of 35 Ct, administered between 0 and 3 days prior to the gathering. Full details are given in S1 Text and S7A Fig.

Next, we shifted attention from the individual to the gathering. We estimated the number of individuals who would be expected to arrive at a 1,000-person gathering while infectious given each testing strategy (40-Ct limit of detection with 1% false negative rate; 35-Ct limit of detection with 5% false negative rate) assuming a 2% prevalence of PCR-detectable infection in the population. To do so, we again drew 1,000 individual-level viral concentration trajectories from the fitted model and drew onset-of-detectability times according to a random uniform distribution from the range of possible times that would allow for the person to have detectable virus (Ct < 40) during the gathering. We counted the number of people who would have been infectious at the gathering (a) in the absence of testing and (b) given a test administered between 0 and 3 days prior to the gathering. As before, we assumed that the infectiousness threshold corresponded to a Ct value of 30 for the main analysis and considered infectiousness thresholds of 35 Ct and 20 Ct in a supplemental analysis. Full details are given in S1 Text and S7B Fig. To facilitate the exploration of different scenarios, we have generated an online tool (https://stephenkissler.shinyapps.io/shiny/) where users can input test and population characteristics and calculate the effective sensitivity and expected number of infectious individuals at a gathering.

Results

Of the 46 individuals with acute infections, 13 reported symptoms at the time of diagnosis; the timing of the onset of symptoms was not recorded. The median number of positive tests for the 46 individuals was 3 (IQR 2, 5). The minimum recorded Ct value across the 46 individuals had mean 26.4 (IQR 23.2, 30.4). The recorded Ct values for the acute infections with individual-level piecewise linear regressions are depicted in Fig 2.

thumbnail
Fig 2. Reported cycle threshold (Ct) values with individual-level piecewise linear fits.

Ct values (points) for the 46 acute infections aligned by the date when the minimum Ct was recorded for each individual. Lines depict the best-fit piecewise linear regression lines for each individual with breakpoint at day 0. Red points/lines represent individuals who reported symptoms, and blue points/lines represent individuals who did not report symptoms. Five positive tests were omitted that occurred >20 days prior to the individual’s minimum Ct value, all of which had Ct > 35. The vertical axis on the right-hand side gives the conversion from Ct values to RNA concentration. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/Fig2 [10].

https://doi.org/10.1371/journal.pbio.3001333.g002

Based on the viral trajectory model, the mean peak Ct value for symptomatic individuals was 22.3 (95% credible interval [CI] 19.3, 25.3), the mean duration of the proliferation phase was 3.4 days (95% CI 2.0, 4.8), and the mean duration of clearance was 10.9 days (95% CI 7.9, 14.4) (Fig 3). This compares with 22.3 Ct (95% CI 20.0, 24.4), 3.5 days (95% CI 2.5, 4.5), and 7.8 days (95% CI 6.1, 9.7), respectively, for individuals who did not report symptoms at the time of diagnosis (Fig 3). This yielded a slightly longer overall duration of acute infection for individuals who reported symptoms (14.3 days [95% CI 11.0, 17.7]) versus those who did not (11.2 days [95% CI 9.4, 13.4]). For all individuals, regardless of symptoms, the mean peak Ct value, proliferation duration, clearance duration, and duration of acute shedding were 22.3 Ct (95% CI 20.5, 23.9), 3.3 days (95% CI 2.5, 4.2), 8.5 days (95% CI 6.9, 10.1), and 11.7 days (95% CI 10.1, 13.6) (S8 Fig). A full list of the model-inferred viral trajectory parameters is reported in Table 1. There was a substantial amount of individual-level variation in the peak Ct value and the proliferation and clearance stage durations (S9S14 Figs).

thumbnail
Fig 3. Peak cycle threshold (Ct) value and infection stage duration distributions according to symptoms reported at time of diagnosis.

Posterior distributions obtained from 2,000 simulated draws from the posterior distributions for mean peak Ct value (A), mean duration of the proliferation stage (first potential infection detectability to peak Ct) (B), mean duration of the clearance stage (peak Ct to resolution of acute RNA shedding) (C), and total duration of acute shedding (D) across the 46 individuals with an acute infection. The distributions are separated according to whether the person reported symptoms (red, 13 individuals) or did not report symptoms (blue, 33 individuals). The mean Ct trajectory corresponding to the mean values for peak Ct, proliferation duration, and clearance duration for symptomatic versus asymptomatic individuals is depicted in (E) (solid lines), where shading depicts the 90% credible intervals. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv [10].

https://doi.org/10.1371/journal.pbio.3001333.g003

thumbnail
Table 1. Viral dynamic parameters, overall and separated by reported symptoms.

https://doi.org/10.1371/journal.pbio.3001333.t001

Using the full dataset of 68 individuals, we estimated the frequency with which a given Ct value was associated with an acute infection (i.e., the proliferation or clearance phase, but not the persistence phase) and, if so, the probability that it was associated with the proliferation stage alone. The probability of an acute infection increased rapidly with decreasing Ct (increasing viral load), with Ct < 30 virtually guaranteeing an acute infection in this dataset (Fig 4A). However, a single Ct value provided little information about whether an acute infection was in the proliferation or the clearance stage (Fig 4B). This is unsurprising since the viral trajectory must pass through any given value during both the proliferation and the clearance stage. With roughly uniform sampling over time, a given Ct value is more likely to correspond to the clearance stage simply because the clearance stage is longer.

thumbnail
Fig 4. Relationship between single/paired cycle threshold (Ct) values and infection stage.

Probability that a given Ct value lying within a 5-unit window (horizontal axis) corresponds to an acute infection (A and C) or to the proliferation phase of infection assuming an acute infection (B and D). (A) and (B) depict the predictive probabilities for a single Ct value, while (C) and (D) depict the predictive probabilities for a positive test paired with a subsequent test with either lower (red) or higher (blue) Ct. The curves are locally estimated scatterplot smoothing (LOESS) curves to better visualize the patterns. Error bars represent the 90% Wald confidence interval. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/Fig4 [10].

https://doi.org/10.1371/journal.pbio.3001333.g004

We assessed whether a second test within 2 days of the first could improve these predictions. A positive test followed by a second test with lower Ct (higher viral RNA concentration) was slightly more likely to be associated with an active infection than a positive test alone (Fig 4C), and was much more likely to be associated with the proliferation phase than with the clearance phase (Fig 4D).

We next estimated how the effective sensitivity of a pre-event screening test declines with increasing time to the event. For a test with a limit of detection of 40 Ct and a 1% chance of sampling error, the effective sensitivity declines from 99% when the test coincides with the start of the event to 76% when the test is administered 2 days prior to the event (Fig 5A), assuming a threshold of infectiousness at 30 Ct [11]. This 2-day-ahead sensitivity is slightly lower than the effective sensitivity of a test with a limit of detection at 35 Ct and a 5% sampling error administered 1 day before the event (82%), demonstrating that limitations in testing technology can be compensated for by reducing turnaround time. Using these effective sensitivities, we estimated the number of infectious individuals who would be expected to arrive at a gathering with 1,000 people given a pre-gathering screening test and a 2% prevalence of infectiousness in the population. Just as the effective sensitivity declines with time to the gathering, the predicted number of infectious individuals rises with time to the gathering (Fig 5B) since longer delays between the screening test and the gathering make it more likely that an individual’s infection will be undetectable at the time of testing but the individual will be infectious at the time of the event. Changing the infectiousness threshold modulates the magnitude of the decline in effective sensitivity associated with longer testing delays; however, the overall pattern is consistent (S18 Fig).

thumbnail
Fig 5. Effective sensitivity and expected number of infectious attendees at an event, for tests with varying sensitivity.

(A) Effective sensitivity for a test with limit of detection of 40 Ct and 1% sampling error probability (red) and limit of detection of 35 Ct and 5% sampling error probability (blue). (B) Number of infectious individuals expected to attend an event of size 1,000 assuming a population prevalence of 2% infectious individuals for a test with limit of detection of 40 Ct and 1% sampling error probability (red) and limit of detection of 35 Ct and 5% sampling error probability (blue). Shaded bands represent 90% prediction intervals generated from the quantiles of 1,000 simulated events and capture uncertainty both in the number of infectious individuals who would arrive at the event in the absence of testing and in the probability that the test successfully identifies infectious individuals. The dashed line depicts the expected number of infectious individuals who would attend the gathering in the absence of testing. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/Fig5 [10].

https://doi.org/10.1371/journal.pbio.3001333.g005

Discussion

We provide to our knowledge the first comprehensive data on the early-infection RT-qPCR Ct dynamics associated with SARS-CoV-2 infection. We found that viral titers peak quickly, normally within 3 days of the first possible RT-qPCR detection, regardless of symptoms. Our findings highlight that repeated PCR tests can be used to infer the stage of a patient’s infection. While a single test can inform on whether a patient is in the acute or persistent viral RNA shedding stage, a subsequent test can help identify whether viral RNA concentrations are increasing or decreasing, thus informing clinical care. For example, patients near the beginning of their infection may need to be isolated for different amounts of time than patients near the end of their infection. For patients at risk for complications, closer monitoring and more proactive treatment may be preferred for patients near the start of infection than for those who are already nearing its resolution. We also show that the effective sensitivity of pre-event screening tests declines rapidly with test turnaround time due to the rapid progression from detectability to peak viral titers. Due to the transmission risk posed by large gatherings [12], the trade-off between test speed and sensitivity must be weighed carefully. Our data offer to our knowledge the first direct measurements capable of informing such decisions.

Our findings on the duration of SARS-CoV-2 viral RNA shedding expand on and agree with previous studies [1315] and with observations that peak Ct does not differ substantially between symptomatic and asymptomatic individuals [3]. While previous studies have largely relied on serial sampling of admitted hospital patients, our study used prospective sampling of ambulatory infected individuals to characterize complete viral dynamics for the presymptomatic stage and for individuals who did not report symptoms. This allowed us to assess differences between the viral RNA proliferation and clearance stages for individuals with and without reported symptoms. The similarity in the early-infection viral RNA dynamics for symptomatic and asymptomatic individuals underscores the need for SARS-CoV-2 screening regardless of symptoms. The progression from a negative test to a peak Ct value 2–4 days later aligns with modeling assumptions made in various studies [5,16] to evaluate the potential effectiveness of frequent rapid testing programs, strengthening the empirical bases for their findings. Taken together, the dynamics of viral RNA shedding substantiate the need for frequent population-level SARS-CoV-2 screening and a greater availability of diagnostic tests.

The statistical model we developed to infer the viral trajectory parameters is phenomenological: It assumes an exponential increase in viral RNA concentration followed by an exponential decay but does not explicitly encode a biological mechanism leading to these exponential rates and the transition between them. Similar phenomenological models have been used to study the viral dynamics of HIV [17]. More biologically explicit mechanistic models have been used to study SARS-CoV-2 [18,19], but these remain in the early stages of development due to the limited amount of data available to inform such models. Since our primary interest is in the public health implications of SARS-CoV-2 viral trajectories with different magnitudes and durations, a phenomenological model is suitable and has the advantage of being straightforward to implement. The data presented here could be used to parameterize detailed mechanistic models as well, from which further biological insights about SARS-CoV-2 might be gained.

Our findings are limited for several reasons. The sample size is small, especially with respect to symptomatic acutely infected individuals. The cohort does not constitute a representative sample from the population, as it was a predominantly male, healthy, young population inclusive of professional athletes. Viral trajectories may differ for individuals who have been vaccinated or who have been infected with different SARS-CoV-2 variants, which we were unable to assess due to the time frame of our study. Some of the trajectories were sparsely sampled, limiting the precision of our posterior estimates. Symptom reporting was imperfect, particularly after initial evaluation, as follow-up during the course of the disease was not systematic for all individuals. As with all predictive tests, the probabilities that link Ct values with infection stages (Fig 4) pertain to the population from which they were calibrated and do not necessarily generalize to other populations for which the prevalence of infection and testing protocols may differ. Still, we anticipate that the central patterns will hold across populations: first, that low Ct values (<30) strongly predict acute infection and, second, that a follow-up test collected within 2 days of an initial positive test can substantially help to discern whether a patients are closer to the beginning or the end of their infection. Our study did not test for the presence of infectious virus, though previous studies have documented a close inverse correlation between Ct values and culturable virus [11]. Our assessment of pre-event testing assumed that individuals become infectious immediately upon passing a threshold and that this threshold is the same for the proliferation and for the clearance phase. In reality, the threshold for infectiousness is unlikely to be at a fixed viral concentration for all individuals and may be at a higher Ct/lower viral concentration during the proliferation stage than during the clearance stage. Further studies that measure culturable virus during the various stages of infection and that infer infectiousness based on contact tracing combined with prospective longitudinal testing will help to clarify the relationship between viral concentration and infectiousness.

To manage the spread of SARS-CoV-2, we must develop novel technologies and find new ways to extract more value from the tools that are already available. Our results suggest that integrating the quantitative viral RNA trajectory into algorithms for clinical management could offer benefits. The ability to chart patients’ progress through their infection underpins our ability to provide appropriate clinical care and to institute effective measures to reduce the risk of onward transmission. Marginally more sophisticated diagnostic and screening algorithms may greatly enhance our ability to manage the spread of SARS-CoV-2 using tests that are already available.

Supporting information

S1 Fig. Distribution of intervals between consecutive tests.

Histogram of the proportion of consecutive tests that are within n days of one another up to n = 12 days. Only 12 of 2,343 intervals (0.05%) exceeded 12 days. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/FigS1.

https://doi.org/10.1371/journal.pbio.3001333.s001

(PDF)

S2 Fig. Observed Ct values from the study participants (1/4).

Points depict observed Ct values, which are connected with lines to better visualize patterns. Individuals with presumed acute infections are in red. All others are in black. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/data.

https://doi.org/10.1371/journal.pbio.3001333.s002

(PDF)

S3 Fig. Observed Ct values from the study participants (2/4).

Points depict observed Ct values, which are connected with lines to better visualize patterns. Individuals with presumed acute infections are in red. All others are in black. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/data.

https://doi.org/10.1371/journal.pbio.3001333.s003

(PDF)

S4 Fig. Observed Ct values from the study participants (3/4).

Points depict observed Ct values, which are connected with lines to better visualize patterns. Individuals with presumed acute infections are in red. All others are in black. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/data.

https://doi.org/10.1371/journal.pbio.3001333.s004

(PDF)

S5 Fig. Observed Ct values from the study participants (4/4).

Points depict observed Ct values, which are connected with lines to better visualize patterns. Individuals with presumed acute infections are in red. All others are in black. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/data.

https://doi.org/10.1371/journal.pbio.3001333.s005

(PDF)

S6 Fig. A theoretical Ct trajectory.

E[Ct] is the expected Ct value on a given day. The Ct begins at the limit of detection, then declines from the time of infection (to) to the peak at χ cycles below the limit of detection at time tp. The Ct then rises again to the limit of detection after tr days. The model incorporating these parameter values used to generate this piecewise curve is given in the equation for E[Ct(t)] in S1 Text (Supplemental Methods, under the heading “Model fitting”).

https://doi.org/10.1371/journal.pbio.3001333.s006

(PDF)

S7 Fig. Schematics illustrating calculations for effective sensitivity for the expected number of infectious attendees at a gathering, given a pre-gathering test.

(A) To calculate the effective sensitivity of a test intended to screen infectious individuals before a gathering, we first drew 1,000 viral trajectories as defined by the peak Ct, proliferation time, and clearance time from the fitted model (step 1, with 3 draws illustrated in red, green, and blue). We restricted to only individuals with viral concentrations above the infectiousness threshold (here the threshold is at Ct = 30, requiring us to omit the fourth entry). Then, we assigned detectability onset times—i.e., the times at which the trajectories could first be detected by PCR with limit of detection at 40 Ct—according to a standard uniform distribution, ensuring that the trajectories surpassed the infectiousness threshold at some point during the gathering (step 2). The onset times are depicted as colored dots. Finally, for a test administered some span of time prior to the event, we calculated the fraction of these infections the test would detect—this is the effective sensitivity (step 3). For a test administered at the time marked by the vertical black bar, the green trajectory would be detected by both PCR and a rapid test, the red trajectory would be detected by PCR but not a rapid test, and the blue trajectory would not be detected by either test. (B) To calculate the number of people who would arrive at a gathering while infectious, we performed a similar procedure. First, given a gathering size N and prevalence of PCR-detectable individuals p, we drew η trajectories from the fitted model where η ~ Binomial(N, p). Three such draws are depicted in step 1; note that, here, the only requirement was that the individuals were detectable (not necessarily infectious) at the time of the gathering, and so the previously omitted value could now be chosen. Then, as before, detectability onset times (colored dots) were drawn from a uniform distribution ensuring that the individuals were PCR-detectable at the time of the gathering (2). Finally, in step 3, the number of infectious individuals who would attend the gathering in the absence of a pre-gathering test were counted (in this case just the blue trajectory) as well as the number of individuals who would attend the event given a pre-gathering test. Here, the blue trajectory would be detected by a PCR test but not a rapid test at the test time depicted by the vertical black bar. The purple trajectory would be detected by both a rapid test and a PCR test, yet it would not have been infectious at the gathering (in fact, this trajectory never surpasses the infectiousness threshold depicted here). The green trajectory would not be detected by either test but also would not have arrived at the gathering while infectious since it has a relatively late onset time. Repeating this procedure for many simulated gatherings gives an estimate of the expected number of infectious people who would arrive at a gathering given a pre-gathering testing protocol.

https://doi.org/10.1371/journal.pbio.3001333.s007

(PDF)

S8 Fig. Mean peak Ct value and distributions of the proliferation stage, clearance stage, and acute infection duration for individuals with acute infections.

Posterior distributions obtained from 10,000 posterior draws from the distributions for peak Ct value (A), duration of the proliferation stage (infection detection to peak Ct) (B), duration of the clearance stage (peak Ct to resolution of acute RNA shedding) (C), and total duration of acute shedding (D) across the 46 individuals with a verified infection. The mean Ct trajectory corresponding to the mean values for peak Ct, proliferation duration, and clearance duration is depicted in (E) (solid lines), where shading depicts the 90% credible interval. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_combined.csv.

https://doi.org/10.1371/journal.pbio.3001333.s008

(PDF)

S9 Fig. Posterior peak Ct value distributions for the 46 individuals with acute infections.

Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv.

https://doi.org/10.1371/journal.pbio.3001333.s009

(PDF)

S10 Fig. Posterior distributions for the duration of the proliferation stage for 46 individuals with acute infections.

Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv.

https://doi.org/10.1371/journal.pbio.3001333.s010

(PDF)

S11 Fig. Posterior distributions for the clearance stage duration for 46 individuals with acute infections.

Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv.

https://doi.org/10.1371/journal.pbio.3001333.s011

(PDF)

S12 Fig. Best-fit Ct trajectories for the 46 individuals with acute infections.

Thin grey lines depict 500 sampled trajectories. Points represent the observed data, with symptomatic individuals represented in red and asymptomatic individuals in blue. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv (lines) and https://github.com/gradlab/CtTrajectories/tree/main/data (points).

https://doi.org/10.1371/journal.pbio.3001333.s012

(PDF)

S13 Fig. Individual-level peak Ct value and distribution of the proliferation stage, clearance stage, and acute infection duration.

Histograms (grey bars) of 10,000 posterior draws from the distributions for peak Ct value (A), time from onset to peak (B), time from peak to recovery (C), and total duration of infection (D) across the 46 individuals with an acute infection. Grey curves are kernel density estimators to more clearly exhibit the shape of the histogram. Black curves represent the best-fit normal (A) or gamma (B–D) distributions to the histograms. The duration of infection is the sum of the time from onset to peak and the time from peak to recovery. The best-fit normal distribution to the posterior peak Ct value distribution had mean 22.3 and standard deviation 4.2. The best-fit gamma distribution to the proliferation stage duration had shape parameter 2.3 and inverse scale parameter 0.7. The best-fit gamma distribution to the clearance stage duration had shape parameter 2.4 and inverse scale parameter 0.3. The best-fit gamma distribution to the total duration of infection had shape parameter 4.3 and inverse scale parameter 0.4. Alternatively, the proliferation, clearance, and total duration of infection distributions can be summarized as log-normal distributions. The best-fit log-normal distribution to the proliferation stage duration had location parameter μ = 0.93 and scale parameter σ = 0.82. The best-fit log-normal distribution to the clearance stage duration had location parameter μ = 1.9 and scale parameter σ = 0.83. The best-fit log-normal distribution to the total duration of infection had location parameter μ = 2.3 and scale parameter σ = 0.53. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv.

https://doi.org/10.1371/journal.pbio.3001333.s013

(PDF)

S14 Fig. Peak viral concentration and overall posterior viral concentration trajectories in terms of genome equivalents per milliliter.

Posterior peak viral concentration distribution for symptomatic (red) and asymptomatic (blue) individuals (A) and for all individuals combined (B). Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_split.csv (A) and https://github.com/gradlab/CtTrajectories/tree/main/output/params_df_combined.csv (B).

https://doi.org/10.1371/journal.pbio.3001333.s014

(PDF)

S15 Fig. Ct values from the Yale and Florida labs.

Points depict the Ct values for SARS-CoV-2 nasal swab samples that were tested in both the Florida and Yale labs. Ct values from Florida represent Target 1 (ORF1ab) on the Roche cobas system, and Ct values from Yale represent N1 in the Yale multiplex assay. The solid black line depicts the best-fit linear regression (intercept = −6.25, slope = 1.34, R2 = 0.86). The dashed black line marks the 1–1 line where the points would be expected to fall if the 2 labs produced identical results. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/FigS15.

https://doi.org/10.1371/journal.pbio.3001333.s015

(PDF)

S16 Fig. Residuals from the Yale/Florida Ct regression.

Points depict the residual after removing the best-fit linear trend in the relationship between the Yale and Florida Ct values. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/FigS16.

https://doi.org/10.1371/journal.pbio.3001333.s016

(PDF)

S17 Fig. Quantile–quantile plot of the residuals from the Yale/Florida Ct regression.

The residuals were standardized (by subtracting the mean of all residuals from each residual and then dividing each residual by the standard deviation of all residuals) before being compared with the theoretical quantiles of a normal distribution with mean 0 and standard deviation 1. The points depict the empirical quantiles of the data points, and the line depicts where the points would be expected to fall if they were drawn from a standard normal distribution. Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/FigS17.

https://doi.org/10.1371/journal.pbio.3001333.s017

(PDF)

S18 Fig. Effective sensitivity and expected number of infectious attendees at a gathering, given a pre-gathering test and varying infectiousness thresholds.

(A and C) Effective sensitivity and (B and D) number of infectious individuals expected to attend a gathering of size 1,000 assuming a population prevalence of 2% infectious individuals and a test with limit of detection of 40 Ct and 1% sampling error probability (red) and a test with limit of detection of 35 Ct and 5% sampling error probability (blue) administered between 0 and 3 days before the gathering. For (A) and (B) individuals are assumed to be infectious when their Ct value is below 35. For (C) and (D) individuals are assumed to be infectious when their Ct value is below 20. Shaded bands represent 90% prediction intervals generated from the quantiles of 1,000 simulated events and capture uncertainty both in the number of infectious individuals who would arrive at the event in the absence of testing and in the probability that the test successfully identifies infectious individuals. The dashed lines in (B) and (D) depict the expected number of infectious individuals who would attend the gathering in the absence of testing. Setting the infectiousness threshold at higher viral concentration (20 Ct versus 35 Ct) makes it less likely that an individual will become infectious at all during the course of their acute infection, leading to the lower expected number of infectious individuals at the gathering in (D) versus (B). Underlying data are available at https://github.com/gradlab/CtTrajectories/tree/main/figure_data/FigS18.

https://doi.org/10.1371/journal.pbio.3001333.s018

(PDF)

S19 Fig. Illustration of why effective sensitivity declines more sharply with testing delays for high versus low infectiousness thresholds.

For a given viral trajectory conditioned on infectiousness during a gathering, there is a wider range of possible proliferation onset times when the infectiousness threshold is low (blue) versus when the infectiousness threshold is high (red). Additionally, the range of possible onset times for the low infectiousness threshold versus the high infectiousness threshold is skewed to the left since the clearance time is longer than the proliferation time. Because of this, a low infectiousness threshold makes it easier for a pre-gathering test to pick up a trajectory that would be infectious at the time of the gathering. Conversely, a high infectiousness threshold shortens the window of possible onset times that guarantee infectiousness during the gathering, making it more difficult for a pre-gathering test to detect the trajectory. This is reflected in the steeper decline in the effective sensitivity for a high infectiousness threshold (Ct = 20) than for a low infectiousness threshold (Ct = 35) (see S18A and S18C Fig).

https://doi.org/10.1371/journal.pbio.3001333.s019

(PDF)

S1 Table. Standard curve relationship between genome equivalents and Ct values.

Synthetic T7 RNA transcripts corresponding to a 1,363-base-pair segment of the SARS-CoV-2 nucleocapsid gene were serially diluted from 106 to 100 and evaluated in duplicate with RT-qPCR. The best-fit linear regression of the average Ct on the log10-transformed standard values had slope −3.60971 and intercept 40.93733 (R2 = 0.99).

https://doi.org/10.1371/journal.pbio.3001333.s020

(PDF)

S2 Table. Viral dynamic parameters for sensitivity analysis 1, omitting person 3047.

https://doi.org/10.1371/journal.pbio.3001333.s021

(PDF)

S3 Table. Viral dynamic parameters for sensitivity analysis 2, assuming 95% PCR sensitivity or a 5% probability of false negative.

https://doi.org/10.1371/journal.pbio.3001333.s022

(PDF)

S4 Table. Viral dynamic parameters for sensitivity analysis 3, removing the upper bounds for the proliferation and clearance times.

https://doi.org/10.1371/journal.pbio.3001333.s023

(PDF)

S5 Table. Viral dynamic parameters for sensitivity analysis 4, using “low” priors for the proliferation and clearance times (mean 3.5 and 7.5 days, respectively).

https://doi.org/10.1371/journal.pbio.3001333.s024

(PDF)

S6 Table. Viral dynamic parameters for sensitivity analysis 5, using “high” priors for the proliferation and clearance times (mean 10.5 days and 22.5 days, respectively).

https://doi.org/10.1371/journal.pbio.3001333.s025

(PDF)

Acknowledgments

We thank the NBA, the National Basketball Players Association (NBPA), and all of the study participants who are committed to applying what they learned from sports towards enhancing public health. In particular, we thank D. Weiss of the NBA for his continuous support and leadership. We are appreciative of the discussions from the COVID-19 Sports and Society Working Group. We also thank D. Larremore for comments on the manuscript, J. Hay and R. Niehus for suggestions on the statistical approach, and P. Jack and S. Taylor for laboratory support.

References

  1. 1. Furukawa NW, Brooks JT, Sobel J. Evidence supporting transmission of severe acute respiratory syndrome coronavirus 2 while presymptomatic or asymptomatic. Emerg Infect Dis. 2020;26(7):e201595. pmid:32364890
  2. 2. Tom MR, Mina MJ. To interpret the SARS-CoV-2 test, consider the cycle threshold value. Clin Infect Dis. 2020;71(16):2252–4. pmid:32435816
  3. 3. Walsh KA, Jordan K, Clyne B, Rohde D, Drummond L, Byrne P, et al. SARS-CoV-2 detection, viral load and infectivity over the course of an infection. J Infect. 2020;81(3):357–71. pmid:32615199
  4. 4. Wyllie AL, Fournier J, Casanovas-Massana A, Campbell M, Tokuyama M, Vijayakumar P, et al. Saliva or nasopharyngeal swab specimens for detection of SARS-CoV-2. N Engl J Med. 2020;383(13):1283–6. pmid:32857487
  5. 5. Larremore DB, Wilder B, Lester E, Shehata S, Burke JM, Hay JA, et al. Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance. Sci Adv. 2021;7(1):eabd5393. pmid:33219112
  6. 6. Mack CD, DiFiori J, Tai CG, Shiue KY, Grad YH, Anderson DJ, et al. SARS-CoV-2 transmission risk among National Basketball Association players, staff, and vendors exposed to individuals with positive test results after COVID-19 recovery during the 2020 regular and postseason. JAMA Intern Med. 2021 Apr 22. pmid:33885715
  7. 7. Xiao AT, Tong YX, Zhang S. Profile of RT-PCR for SARS-CoV-2: a preliminary study from 56 COVID-19 patients. Clin Infect Dis. 2020;71(16):2249–51. pmid:32306036
  8. 8. Cleary B, Hay JA, Blumenstiel B, Harden M, Cipicchio M, Bezney J, et al. Using viral load and epidemic dynamics to optimize pooled testing in resource-constrained settings. Science Translational Medicine. 2021 apr 14. pmid:32511487
  9. 9. Carpenter B, Gelman A, Hoffman MD, Lee D, Goodrich B, Betancourt M, et al. Stan: a probabilistic programming language. J Stat Softw. 2017;76(1).
  10. 10. Kissler S. gradlab/CtTrajectories. Zenodo. 2021 Jun 17.
  11. 11. Singanayagam A, Patel M, Charlett A, Lopez Bernal J, Saliba V, Ellis J, et al. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Euro Surveill. 2020;25(32):2001483. pmid:32794447
  12. 12. Cevik M, Marcus J, Buckee C, Smith T. SARS-CoV-2 transmission dynamics should inform policy. SSRN Electron J. 2020 Sep 21. pmid:32964919
  13. 13. Cevik M, Tate M, Lloyd O, Enrico Maraolo A, Schafers J, Ho A. SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis. The Lancet Microbe. 2020 19 Nov. pmid:33521734
  14. 14. Houlihan C, Vora N, Byrne T, Lewer D, Heaney J, Moore DA, et al. SARS-CoV-2 virus and antibodies in front-line Health Care Workers in an acute hospital in London: preliminary results from a longitudinal study. medRxiv. 2020 Jun 9.
  15. 15. Lee S, Kim T, Lee E, Lee C, Kim H, Rhee H, et al. Clinical course and molecular viral shedding among asymptomatic and symptomatic patients with SARS-CoV-2 infection in a community treatment center in the Republic of Korea. JAMA Intern Med. 2020;180(11):1447–52. pmid:32780793
  16. 16. Paltiel AD, Zheng A, Walensky RP. Assessment of SARS-CoV-2 screening strategies to permit the safe reopening of college campuses in the United States. JAMA Netw Open. 2020;3(7):e2016818. pmid:32735339
  17. 17. Huang Y, Dagne GA, Zhou S, Wang Z. Piecewise mixed-effects models with skew distributions for evaluating viral load changes: a Bayesian approach. Stat Methods Med Res. 2015;24(6):730–46. pmid:22045781
  18. 18. Ke R, Zitzmann C, Ribeiro RM, Perelson AS. Kinetics of SARS-CoV-2 infection in the human upper and lower respiratory tracts and their relationship with infectiousness. medRxiv. 2020 Sep 27.
  19. 19. Goyal A, Reeves DB, Cardozo-Ojeda EF, Schiffer JT, Mayer BT. Viral load and contact heterogeneity predict SARS-CoV-2 transmission and super-spreading events. eLife. 2021 Feb 23. pmid:33620317