HOME PAGE OF SEMMEL-WEIS.ORG Semmel Weis in Landeck, Austria with Die Silberspitze (SilverPeak) in the background Ignaz Semmelweis on Hungarian postage stamp Frodo Ring introduces RED BOX WARNING AGAINST KAPLAN-MEIER Grim Reaper introduces the SAN MIGUEL (2008) TRILOGY IS THERE AN ALTERNATIVE TO INFOMERCIALS? THESE FOUR RATS GAVE THEIR LIVES DEMONSTRATING THAT THERE IS AN ALTERNATIVE. Chinese woman holding giant, bamboo-eating rat Bright future for Johnson&Johnson DARZALEX
FDA Agent Badge gets you into THE JOHNSON & JOHNSON RAP SHEEP Dr ROBERT Z ORLOWSKI, one of the MAGNIFICENT EIGHT represented here as a SUPERMAN in the area of multiple myeloma 83% survival at Months=30 is the inflated survival indicated in a misleading Kaplan-Meier graph Dr Paul Richardson wearing the recommended badge which reflects that Big Pharma's power over him has the strength of $19.6 million Best Hospitals ranking by USNews HOW MANY SUBJECTS PER GROUP? Dr SAGAR LONIAL debates Dr Paul Richardson Meletios Dimopoulos
A VERY SPECIAL INTEREST HERE TO SEE YOU William James Mayo Dr Matt Kalaycio unable to dispute payment Incongruity arrow ODOMZO may exhibit a Kalaycio-Boom in a Mayo Clinic Rochester clinical trial having Dr Francis Buadi as Principal Investigator      


by Semmel Weis
First published 25Jan2018  Last edited 25Jan2018  06:50pm

The Dimopoulos (2017) Kaplan-Meier Overall-Survival graph below shows two curves which carry vertical slashes marking the occasion of subjects discontinuing the experiment (being "censored" the researchers call this).

A more inept portrayal of subject-discontinuation information can hardly be imagined.  For one thing, the slashes obstruct the view of the curves themselves.  More importantly, the slashes' appearance does not change as they pile higher on top of each other — for example, would you be able to discern from the appearance of these slashes how many might lie on the blue curve in the vicinity of Months=36 to Months=42?  The answer happens to be 180, suggesting that in places along the blue curve the slashes might be piled 10 or 20 deep.  In clinical-trial graphs like this, one would be hard-pressed to draw any conclusions from the slashes.  Ineffectual though this manner of presenting subject-discontinuation information may be, it is the unobjected-to convention.

What is not at all conventional about the Dimopoulos graph is that it happens to convey much the same censorship information numerically — in the Number-Censored values in parentheses underneath the graph, which is a most welcome innovation, permitting — as will be demonstrated below — a clearer view of what is happening.

It will help to keep in mind that Number-At-Risk means Number-Of-Subjects-Remaining-In-The-Experiment.  As all subjects remaining in the experiment could die, they can all be said to be at risk of dying.

Who is not at risk of dying?  In the first place, the subjects who are already recorded as having died.

And also not at risk are the Censored subjects who have disappeared from the view of the researchers.  The fled and the shed, as I prefer to call them — the fled having themselves decided to quit the experiment, and the shed having been terminated by the researchers, perhaps primarily because the researchers closed down the experiment at a time when late-enrolling subjects had just begun their treatment, or were only part-way through it.  The Censored aren't At Risk of dying in the sense that their departure makes it impossible for the researchers to ever notice that they have died.  In other words, for purposes of the experiment, once fled or shed, a subject can never be found dead.  Such is the thinking of researchers who rely on Kaplan-Meier graphs.

Meletios Dimopoulos (2017) releases 18 more numbers than is conventional

Carfilzomib or bortezomib in relapsed or refractory multiple myeloma (ENDEAVOR): an interim overall survival analysis of an open-label, randomised, phase 3 trial
Dr Prof Meletios A Dimopoulos, MD, Hartmut Goldschmidt, MD, Ruben Niesvizky, MD, Prof Douglas Joshua, MD, Prof Wee-Joo Chng, MD, Albert Oriol, MD, Prof Robert Z Orlowski, MD, Prof Heinz Ludwig, MD, Prof Thierry Facon, MD, Roman Hajek, MD, Katja Weisel, MD, Vania Hungria, MD, Leonard Minuk, MD, Shibao Feng, PhD, Anita Zahlten-Kumeli, PhD, Amy S Kimball, MD, Philippe Moreau, MD
LANCET,  2017, Volume 18, No. 10, pp1327-1337.    The Overall Survival graph below is more directly available at GRAPH.
The drug which comes out looking good in this clinical trial is Carfilzomib, which is marketed under the tradename Kyprolis by Onyx Parmaceuticals, which funded the Dimopoulos (2017) clinical trial.   The drug which comes out looking bad is Bortezomib, which is marketed as Velcade by Millennium Pharmaceuticals in the US and by Johnson & Johnson outside the US, neither of which funded the Dimopoulos (2017) clinical trial.

Dimopoulos (2017) Overall-Survival graph with 18 more numbers than is usual

The innovative and welcome inclusion of Number-Censored values (within parentheses above) permits readers of the article to conduct investigations of their own, starting most simply with calculating the number of subjects who are known to have died at each Months value, which number Dimopoulos knew and could have presented as a third value attached to each pair that he does show, just as we have attached that third value in bold within each cell below.  However, for the sake of clarity, it should be kept in mind that Dimopoulos actually counted the deaths as they took place, whereas we will calculate the numbers of these deaths from the two numbers that are given.

And to the reader who imagines that the number of observed deaths is what the survival curves convey, as for example when the survival curve drops to 80%, then 20% have been observed to die — to such a reader it must be most emphatically objected that Kaplan-Meier survival curves do nothing of the sort.  We will right now calculate exactly how many subjects have been observed to die at the various Months Since Randomization, upon which it will become clear that number of deaths is not what the Kaplan-Meier survival curves tell us.

So, then, how do we calculate the number Dead below, as for example in the Carfilzomib row where at Months=0, none have died, while at Months=6, 34 have died?  Notice first that the Carfilzomib Group starts out with N-Per-Group=464, and that 464 remains the Carfilzomib N-Per-Group forever.  All that changes over time is the status of these 464 subjects, and that status has to be one of three: either At Risk (enrolled and alive), or Censored (no longer enrolled and don't know dead/alive), or Dead.  Right at the beginning of the study, at Months=0, all 464 subjects are At Risk (they're all enrolled in the study and all alive), so of course zero have been Censored, and zero have Died.  Now at Months=6, Dimopolous tells us that only 423 are At Risk (still in the study) and that 7 have been Censored (are no longer in the study), so all the rest must be Dead, which by subtraction is 464-423-7 = 34 Dead.  The simple rule being followed is that in the Carfilzomib row the three numbers within each cell must sum to 464, and in the Bortezomib row the three numbers within each cell must sum to 465.

Number At Risk       (= Number OBSERVED to be still remaining in the experiment, and whose names are known)
Number Censored   (= Number OBSERVED to have disappeared from the experiment, and whose names are known)
Number Dead          (= Number OBSERVED to have died, and whose names are known)

Group Subject Status Months Since Randomization
0 6 12 18 24 30 36 42 48
N-Per-Group = 464
At Risk (given)
Censored (given)
Dead (computed)
N-Per-Group = 465
At Risk (given)
Censored (given)
Dead (computed)

The uppermost number in each cell above appears in RowA in the tables below.  The middle number in each cell above appears in RowC below.  The bottommost number in each cell above, where it is in bold to differentiate it because we have calculated it, appears in RowG below, where it is also shown bold as a reminder of its origins above.  And so we discover that the Dimopoulos addition of the RowC values to the conventionally-offered RowA values has enabled the computation of a much larger number of offshoot variables than just RowG.

As a first example of the benefits to be derived from the inclusion of the Number-Censored values is having been able to tell there were 180 slashes piled on top of each other in a section of the blue curve — which estimate comes from the 86 + 94 to be found in RowD immediately below:

Some Carfilzomib Curves in Dimopoulos (2017)
Variable Name Source Months Since Randomization
0 6 12 18 24 30 36 42 48
A # AtRisk given 464 423 373 335 308 270 162 66 10
B P AtRisk          B(n) = A(n)/464 1.0000 0.9116 0.8039 0.7220 0.6638 0.5819 0.3491 0.1422 0.0216
C # Censored  (cumulative) given 0 7 16 21 25 35 121 215 266
D # Censored  (interval) D(n) = C(n)-C(n-1) 0 7 9 5 4 10 86 94 51
E # UnDead E(n) = A(n)+C(n) 464 430 389 356 333 305 283 281 276
F P UnDead       F(n) = E(n)/464 1.0000 0.9267 0.8384 0.7672 0.7177 0.6573 0.6099 0.6056 0.5948
G # Dead  (cumulative) G(n) = 464-E(n) 0 34 75 108 131 159 181 183 188
H # Died (interval) H(n) = G(n)-G(n-1) 0 34 41 33 23 28 22 2 5

Some Bortezomib Curves in Dimopoulos (2017)
Variable Name Source Months Since Randomization
0 6 12 18 24 30 36 42 48
A # AtRisk given 465 402 351 293 256 228 140 39 5
B P AtRisk          B(n) = A(n)/465 1.0000 0.8645 0.7548 0.6301 0.5505 0.4903 0.3011 0.0839 0.0108
C # Censored  (cumulative) given 0 28 40 50 56 58 130 221 251
D # Censored  (interval) D(n) = C(n)-C(n-1) 0 28 12 10 6 2 72 91 30
E # UnDead E(n) = A(n)+C(n) 465 430 391 343 312 286 270 260 256
F P UnDead       F(n) = E(n)/465 1.0000 0.9247 0.8409 0.7376 0.6710 0.6151 0.5806 0.5591 0.5505
G # Dead  (cumulative) G(n) = 465-E(n) 0 35 74 122 153 179 195 205 209
H # Died (interval) H(n) = G(n)-G(n-1) 0 35 39 48 31 26 16 10 4

Revelations begin to emerge with graphing.

In the Carfilzomib Curve graph immediately below, we start by plotting Carfilzomib RowB , the proportion of subjects at risk, meaning still enrolled in the experiment, and which demarcates the lower boundary of where the survival curve can be placed (the survival curve can never be allowed to tell us that there are fewer subjects alive than the still-enrolled subjects who can be seen walking around the clinic).

And we next plot Carfilzomib RowF , which acts as the upper boundary of where the survival curve can be placed (the survival curve can never be allowed to tell us that there are more subjects alive than the number of Undead, where Undead covers all who have not been proven dead, namely the Living plus the Censored).

Next we lift the Carfilzomib survival curve from the squat (much wider than tall) Dimopoulos (2017) graph at the top of the instant page, reshape that curve to fit the taller-than-wide graph below, star it with ☆s, and label it "As drawn in Dimopoulos".  That starred curve below is not plotted from Dimopoulos-supplied numbers, but from the Dimopoulos-supplied drawn curve.

As is to be expected, in both Carfilzomib (blue) and Bortezomib (red) graphs, the starred Kaplan-Meier Overall-Survival curve does lie within the limits imposed on it, and Kaplan-Meier does place it high within the UNOBSERVED area delineated by these limits, "UNOBSERVED" referring to the censored subjects whom the researchers have lost contact with and so can't tell whether they are dead or alive.  The impression created is that starting at Months=30, the researchers begin stepping up the rate of censoring subjects out of the experiment, thus creating an increasingly-broad UNOBSERVED area, which later allows Kaplan-Meier to step in to place survival curves high within that area.

Two survival curves from Dimopoulos (2017) lying within the minima and maxima dictated by the numbers supplied underneath the graph

One detail supporting the hypothesis of deliberate censorship-acceleration is the near-disappearance of deaths indicated by the flattening of RowF curves between Months=36 and Months=48, which is the opposite of what might be expected.  That is, multiple myeloma is said to be incurable, and the subjects in Dimopoulos (2017) had previously found between one and three lines of therapy having failed them, and so when they next enrolled in Dimopoulos (2017), it was with their multiple myeloma considered to be relapsed or refractory, and on top of that now finding themselves having endured an additional Months=30 to 48 of chemotherapy, which paints a bleak picture of lives that cannot be expected to continue much longer, and yet the Dimopoulos data shows their RowF curves plateauing rather than plunging.

Also supportive of the deliberate-censorship-acceleration hypothesis is the sharp change of direction of the RowB curves at Months=30, which produces a curve shape that is not without precedent, as it resembles the dark-adaptation curve shown in the two graphs below, whose rod-cone break signals that two independent processes take sequential control during dark-adaptation, first the dark-adaptation of the cones, and ten minutes later the dark-adaptation of the rods.

The case of the RowB change of direction may similarly signal that some different process takes over at around Months=30, which different process can be hypothesized to be researchers confronted with a spate of rapidly-failing subjects who if they are allowed to die while still enrolled in the experiment threaten to make both groups look like they bring sudden death to all, quite unlike the Dimopoulos (2017) graph at the top of the instant page which seems to offer hope into the indefinite future.

Dark-adaptation curve
  Dark-adaptation curve

How deliberate-censorship-acceleration might be implemented is not hard to imagine.  A researcher can point out to rapidly-weakening and greatly-suffering subjects that their drug regimen seems to be failing them and that they might consider abandoning the Dimopoulos clinical trial so as to be able to give some alternative therapy a chance; and ultimately when the deaths threaten to become too numerous, terminating the experiment for all subjects — which in either case censors subjects from the OBSERVED-TO-BE-ALIVE (= At Risk) area into the UNOBSERVED (= Censored) area where, thanks to Kaplan-Meier, their deaths will go unrecorded.

This line of thought culminates in five questions:

  1. As the inclusion by Dimopoulos (2017) of Number-Censored values underneath the Overall-Survival graph has enabled the drawing of the clarifying graphs above, and as it may enable the drawing of still other clarifying graphs in the future, shouldn't such inclusion become mandatory underneath all Kaplan-Meier graphs?

  2. As the drawing of the RowB and RowF lower and upper bounds for the Kaplan-Meier Overall-Survival curves reveals their wide separation, and thus the wide range of heights to which a survival curve could conceivably be lifted, should not researchers be obligated to also draw exactly such graphs as the ones above, despite their gaping UNOBSERVED areas tending to encourage suspicion that the Kaplan-Meier curves have been given so much latitude in order to permit them to be placed arbitrarily high?  More importantly, is it not obligatory to disclose such graphs as these to patients and to subjects to clarify and to emphasize that the two boundary curves are all that has actually been observed in research, with the true survival curve known to lie somewhere in between, the Kaplan-Meier location being favored by pharmaceutical manufacturers because it is high, but with a much lower true curve being possible?

  3. As the drawing of vertical slashes on clinical-trial survival curves to mark points of individual censorship creates an unholy mess of no use to anyone, and as the location of censorship is advantageously communicated by the alternative of disclosing censorship numbers, perhaps the drawing of those slashes should be discouraged, at least in cases where they are numerous?

  4. In view of the probability that particularly-interested readers of any clinical trial are likely to be able to do a better job analyzing the data than the original authors, would it not speed the advancement of science to always publish the raw data on the Internet, freely downloadable by anyone?  The Dimopoulos (2017) survival data that particularly interests me, for example, comprises a tiny data package consisting for 992 patients of only four pieces of information: (1) what group the patient is in, either Carfilzomib or Bortezomib, (2) patient's date of randomization, (3) date of termination, and (4) category at termination, either At Risk, Censored, or Dead.  That would be the minimum.  Slightly expanded would include (5) the patient's sex, (6) date of birth, and (7) the Hospital at which the trial was conducted.  In days of yore when such data might have to be copied by hand and analyzed using calculators, and stored on paper in filing cabinets, any re-analysis would have been a great chore, but today the data can be transmitted within seconds, and analyzed within seconds as well, and stored within an area smaller than the period at the end of this sentence.  Therefore, why is not all data for all published research made freely available, at least its core which in the case of Dimopoulos (2017) could be the seven subject attributes listed above?

  5. Although improvements of any kind must always be welcome, such as the improved transparency made possible by Dimopoulos's release of eighteen extra numbers (the eighteen "Number Censored" values printed underneath the Overall-Survival graph), or the greatly-improved analyses that open access to the raw data would enable, is it not unfortunately the case in research such as that of Dimopoulos (2017) that the improvements would contribute nothing toward the understanding of cancer because disregard of methodological requirements has invalidated the underlying data many times over?

One does not have to look beyond the second sentence under the heading "Methods" in Dimopoulos (2017) to find evidence that such a harsh judgment is deserved, in which sentence is disclosed that his 464 + 465 = 929 "patients were recruited from 198 hospitals and outpatient clinics in 27 countries in Europe, North America, South America, and the Asia-Pacific region."

Anyone versed in scientific method will recognize that this is grandstanding to an uneducated audience to whom scientific validity is synonymous with testing vast numbers of subjects in far-flung locations, and to which audience the idea would seem fantastic and incredible that testing as few as 10 subjects in a single hospital could possess the full scientific validity that Dimopoulos (2017) with his 929 subjects in 198 hospitals in 27 countries fully lacks.

I have already addressed you on this issue in my
HOW MANY SUBJECTS PER GROUP? to which I am still awaiting your reply.   ▢