Treating Cancer Beyond the Official Evidence
A recent study examines the oft opaque off-label and off-guideline practices in oncology, demonstrating the possible utility of real-world data.
It is often assumed that medicine is a field of science. Although rarely explicitly stated by those prone to this assumption, this would mean that typical medical scenarios have clinical guidance backed by evidence from properly designed, controlled, and powered clinical trials. This is of course a fairy tale. Physicians and other non-physician practitioners still rely on clinical experience and intuition to fill the many gaps in clinical guidelines and treatment algorithms. It would be ideal to move beyond this state-of-affairs to a completely scientific paradigm.1 Such a future is unlikely ever to arise given the complexity of human biology and the expense of clinical research and drug development. Nonetheless, we should strive for this ideal.
Given this fuzzyness to the practice of medicine, I think it is important to analyze real-world clinical practice, specifically scenarios where physicians have felt compelled to go beyond the confines of FDA drug labels and professional guidelines.2 Unfortunately, it is often challenging to collect such data in a systematic way.3 Some of this has to do with how the American healthcare system is organized along with other regulatory and legal hurdles. This is a pressing issue of medical concern. Research has turned to emerging real-world data (RWD) repositories as a stopgap option. A robust analysis of real-world cancer care comes from a recent study in Cell Reports Medicine. This study has provided a retrospective look into off-label and off-guideline practices in oncology along with an attempt to model determinants of such practices. This work was enabled by the nationwide Flatiron Health (FH) electronic health record (EHR)-derived de-identified database.
Defining Terms and Some Qualifications
Before I jump into coverage of this interesting study, I wanted to define two important terms that I will be using frequently. When I use the term“off-label,” I am referring to a situation where a physician has prescribed a drug in a clinical setting not specified on the label approved by an official regulatory body (e.g. the FDA when talking about U.S. pharmaceuticals). This could mean using a drug in a completely different disease state, e.g. using a drug approved in lung cancer for breast cancer, or a different treatment setting, e.g. using a drug approved for a later line of care in the first line. When I use the term “off-guideline,” I’m referring to situations where a physician has prescribed a drug in a clinical setting not included in evidence-based guidelines published by experts. When it comes to oncology care, the go-to professional guidelines are published by the National Comprehensive Cancer Network (NCCN). Guidelines are especially important to determining payer coverage decisions, meaning if it’s in the guidelines, it is likely the insurance company provides coverage of that option. For the study, I’ll cover today, the authors use a conservative definition of off-label/off-guideline by only counting cases where a drug is used in an entirely different disease state that its approval or recommendation.
Generally, there is significant agreement among label and guidelines. However, discrepancies sometimes arise, meaning there isn’t a one-to-one correspondence between off-label and off-guideline. Some of these discrepancies have simply to do with logistical hurdles like keeping up-to-date with rapidly evolving research and regulatory decision. Other discrepancies concern differing expert opinions on the available research.
The terms “off-label” and “off-guideline” may trigger concerns about greater risks associated with such intervention. This is not necessarily the case nor does it always mean that there is uncertainty about the effectiveness of the intervention. For example, there are drugs like oxaliplatin, a platinum chemotherapy, that are used off-label or off-guideline frequently or even routinely. These situation are often an artefact of the strictures of the process of regulatory approval. In these situations, it is widely known that a given chemotherapy can be expected to provide a benefit across a range of cancers because of the known mechanism and existing research or real-world data, but the evidence simply hasn’t been submitted for approval to the FDA.4
Now, that we have a handle on the terms. I also want to warn readers about the limitations of investing heavily in the conclusions of one study that was performed retrospectively on “real-world data.” The study I’m covering here is observational. It does include some modeling, but the data drawn from the Flatiron Health database may be subject to certain types of biases that may affect some of the conclusions. Furthermore, the sample itself is specific to oncology care in America. Please keep these limitations in mind. Try not to reflexively extend the conclusions of this study of all off-label/off-guideline medicine or even as perfectly representative of oncology care in America. Nonetheless, I think the study is illuminating and provides an important real-world snapshot of the practice of medicine in situ.
Off-Label/Guideline Treatment by the Numbers
One of the impressive features of the Cell Reports Medicine study is its sheer size and detail. The cohort is comprised of 165,912 cancer patients from 280 clinic (~800 sites of care) across the United States. There are a total of 14 different types of cancer included: ~49k patients with advanced non-small cell lung cancer (aNSCLC), ~23k patients with metastatic colorectal cancer (mCRC), ~23.5k patients with metastatic breast cancer (mBC), ~11k patients with multiple myeloma (MM), ~9k patients with chronic lymphocytic leukemia (CLL), and ~8k patients with metastatic pancreatic cancer (mPCa), ~5k patients with ovarian cancer (OC), ~6k adult patients with acute myeloid leukemia (AML), ~6.5k patients with small cell lung cancer (SCLC), ~6k patients with advanced bladder cancer (aBCa), ~6k patients with diffuse large B-cell lymphoma (DLBCL), ~6k patients with advanced melanoma (aMel), ~3k patients with follicular lymphoma (FL), and ~3k patients with hepatocellular carcinoma (HCC). A total of 241 different anti-cancer drugs were used in the study. There were also associated predictive biomarker data derived from Foundation Medicine’s comprehensive genomic profiling (CGP) assays, which rely on next-generation sequencing (NGS) of biopsied cancer tissue.
Among the cohort, 18.6% were treated with at least one off-label drug and 4.4% of were treated with at least one off-guideline drug. Patients with multiple primary cancer diagnoses were more likely to receive off-label and/or off-guideline treatments. These figures fall near the low-end of previously reported ranges. The most commonly used off-label treatments were unsurprisingly chemotherapy drugs: carboplatin, etoposide, and bendamustine. Contrastingly, the most commonly administered off-guideline treatments were more varied by drug type including a gonadotropin-releasing hormone (GnRH) agonist (leuprolide), a chemotherapy (pemetrexed), and a monoclonal antibody (bevacizumab). Physicians turned to off-label usage in small cell lung cancer (SCLC), where treatment options were borrowed from FDA approvals in advanced non-small cell lung cancer (aNSCLC) and ovarian cancer (OC). Physicians turned to off-guideline treatments most often in breast cancer, also borrowing from recommendations in aNSCLC and OC.
Increased Acceptance of Precision Oncology Paradigm May Increase Off-Guideline Treatment
The first sub-analysis run by the study authors evaluates temporal trends in off-guideline treatment. They argue that off-guideline treatments is more interesting and thus foreground it (versus off-label treatments) because off-guideline treatments generally have less of an evidentiary basis. There isn’t much that jumps out from their temporal analysis except that particularly challenging diseases (mCRC, AML, DLBCL, FL, and HCC) have seen more off-guideline treatment. Some of this off-guideline treatment can be explained by use of targeted therapies for unapproved disease states that harbor targetable biomarkers. For instance, binimetinib, a drug approved for BRAF mutation-positive melanoma, has seen increased use in BRAF mutation-positive mCRC.5 This suggests an increased comfort with viewing cancer as a molecular rather than tissue based disease, especially in advanced settings. In other words, the tumor agnostic approach advocated for by precision oncology, which refers to the practice of sequencing the DNA of a tumor and prescribing drugs that target driver mutations, has gained a greater foothold in everyday cancer treatment. In other cases, oncologists are practicing according to anticipated guideline changes based on promising early data. For example ruxolitinib, a FDA approved JAK2 inhibitor for myelofibrosis, has seen increased off-guideline use in AML due to positive data in early clinical trials.
On the other side of the ledger, off-guideline treatment has declined for some challenging diseases too: CLL, SCLC, aBCa, and aMel. Advanced breast cancer (aBCa) saw a sharp drop from 10% to 2% between 2015 and 2020, which is explained by the approval and use of pembrolizumab, the top blockbuster immunotherapy drug. In this setting, pembrolizumab largely replaced the off-guideline use of pemetrexed.
Early Immunotherapy Adoption is Popular. Perhaps too Popular.
Immunotherapy drugs are quite popular in oncology for good reason. There are some remarkable responses that have been observed using them. However, there are some concerns about overuse or reflexive use in settings where they’re unlikely to benefit patients. The FH database affords special insight into real-world immunotherapy usage in such settings. As expected, the researchers found that off-label immunotherapy use is quite prevalent, occurring 7.6% of the time immunotherapy is used. Answering the question of whether this is actually benefitting or harming such patients is challenging and likely beyond the FH database, nonetheless the authors tried their best to provide some insights.
To understand a bit more about the clinical impact of these choices, the research team performed a case study of the off-label usage of nivolumab, another popular immunotherapy drug, in their dataset. They found 8.3% of SCLC patients received nivolumab prior to its approval for SCLC in August 2018. The authors also found that nivolumab usage in off-label setting periodically ran ahead of even clinical trial data, meaning the physicians are essentially prescribing on the basis of clinical performance borrowed from other settings. The aggressive use of immunotherapy did not concentrated in academic medical centers or regions, suggesting a generalized enthusiasm. These trends are potentially concerning, but the authors were not able to clearly demonstrate clinical harm associated with any of these practices. They did create a synthetic study of subsequent line off-label nivolumab versus approved chemotherapy regimens (e.g. gemcitabine, paclitaxel) for second line or greater treatment of metastatic prostate cancer (mPCa) and found that nivolumab doesn’t provide an edge in clinical benefit. It’s good to see that off-label nivolumab is not apparently harming patients, but it’s likely too expensive an option to be favored twice over the just as effective chemotherapy. This case study wasn’t able to address how much consideration of side effects figured into the preference for nivolumab.
Although the FH database was not able to provide clear insights on whether immunotherapy drugs are generally being overused in off-label settings, the study was able to confirm findings from clinical trials that led to the removal of an immunotherapy combo regimen (nivolumab and ipilimumab) from the SCLC NCCN Guidelines in 2021. They found that SCLC patients receiving the nivolumab/ipilimumab combo did not show improved survival versus SCLC patients receiving nivolumab alone in second or greater lines of treatment. Despite not being original, this was a great proof-of-concept finding that demonstrates the potential utility of real-world data.6
Predicting Off-Guideline Treatment
The authors fit a logistic regression model with patients’ characteristics to predict whether or not they received off-guideline treatments. They found that later treatment line, younger age, worse disease (measured by ECOG performance status), and treatment at an academic medical center predicted more off-guideline treatment, while insurance coverage, racial identity, and testing positive for a targetable biomarker like EGFR mutation, BRAF mutation, or HER2 positivity predicted less off-guideline treatment. The biomarker finding contrasts a bit with what was observed in the analysis of temporal trends though the relevant disease-biomarker-drug associations differ, suggesting heterogeneity to the relationship between precision approaches and whether physicians are staying aligned with clinical guidance. This heterogeneous relationship is easy to see in their modeling given how triple-negative status predicts a much greater likelihood of off-guideline treatment. Apart from the biomarker statuses, the other relationships are aligned with prior expectations. Generally, it would be expected that young, more affluent individuals with more serious disease or disease that has been subject to more treatment lines are going to be more likely to get a treatment beyond the guidelines. These are clinical situations where a patient is motivated to try something, has likely exhausted other options yet has the ability to access and tolerate non-standard treatment.
The Bittersweet State of Real-World Data
I wanted to call attention to this study in part because I think it highlights just how much data our current system neglects. FH rendered high quality data for 160k cancer patients, and this wasn’t easy and the findings, while significant, are not incredibly high impact. But what insights could be pulled out of a dataset a log-fold larger? Why don’t we already have such datasets!? In a perfect world, academic medical centers and EHR providers would have already have robust solutions for real-world data (RWD) collection, analysis, and sharing, and the FDA would have exceedingly clear guidance for how RWD could be leveraged to produce evidence that could support future approval decisions or post-market analyses in streamlined fashion. Instead, we have a fragmented, disorganized, and confusing situation. For this study, it took a startup company started in 2012 to fill part of the need.7 The work is a great demonstration of what can be unearthed with high-quality RWD, but the existence of such a study in 2024 is bittersweet in that it highlights the challenges that exist in the RWD space.
The term of art in medicine today is evidence-based medicine (EBM). This term has been around awhile (roughly 3 decades) and is largely what guides medicine today. There still remains quite a bit of angst about what it means to be doing evidence-based medicine including whether medicine today meeting this standard and whether the model as implemented is sustainable and effective.
I’m not making the blanket assertion that the existence of an approved FDA indication or a recommendation from a professional guideline represents a situation in which gold-standard clinical evidence exists, however, for these purposes, we can assume that there is some evidence of a clinical benefit has been demonstrated. There is of course debate about whether the FDA and expert panels are drawing the proper conclusions about scientific evidence. These types of debates can be followed in the medical literature or at secondary/tertiary sources like the Substack The Drug Development Letter from Vinay Prasad.
Some of the existing literature reviews of the subject acknowledge a surprising amount of uncertainty, quoting enormous ranges. For instance, a 2017 literature review of ~200 studies of off-label practices of patients from different countries found 13%–71% of adult patients with cancer received at least one off-label chemotherapy. Further, that same review found that this off-label drug use was outside standard treatment guidelines in 7%-31% of cases.
The application process can be quite costly. In fact, about half of the FDA’s budget is from application fees. For older type drugs with existing approvals, there is little to no incentive for a maker to eat the costs of an application simply to expand the label. Guidelines can often capture this type of thing. For instance, carboplatin doesn’t have FDA approval for mBC, SCLC, aBCa, DLBCL, aMel, and FL, but is recommended by NCCN guidelines. Whether they do depends on a number of factors.
There is a similar drug in the same class as binimetinib that is approved across all tumor types with BRAF mutations, trametinib.
Outside of IO drugs, the authors do this as well for the use of off-label hormone therapy on subsets of patients with OC.
Although successful, it isn’t like the RWD/RWE industry is going gangbusters either. FH has accomplished a lot by assembling a huge high quality dataset but turning that data into real clinical insight that turn a profit is another thing. Quick history of FH: They were birthed in 2012, acquired in 2018 by Roche in 2018 for ~$2B, and now Roche is looking to sell.
The disjointed and incomplete picture garnered from clinical trial data so eloquently detailed by Dr. Thacker in his post bears particular relevance in the case of oncology drugs, which address a disease with a vast array of underlying molecular pathologies that are specific to each patient. There is no way to systematically evaluate the safety and efficacy of medicines that are used for a disease state characterized by varying levels of metabolic, genetic, immunological, and physiological dysregulation.
The ”secret sauce” of cancer treatment is to find the right drug for the patient at the right time of their disease. It sounds far easier than it is, and for good reason: cancer is the most complex of human ailments. Due to the unique genetic background of each patient, and the specific history of their cancer, each patient’s disease is a “one-of-a-kind” affair. There is no way to systematically evaluate the safety and efficacy of oncology medicines across wide populations without considering the nature of the cancer itself. This is why precision oncology is so desperately needed to address each patient's cancer on a case-by-case basis.
But it isn’t hopeless. Far from it. While precision oncology is just in its infancy, there is little doubt that by applying our burgeoning capabilities in genetic sequencing and AI-based modeling using huge datasets, we can now determine which patients are most likely to benefit from specific medicines. In many (hopefully most) cases we can genetically determine if the target of a therapeutic are present and playing a role in disease etiology.
When I started graduate school in the late 70’s, I don’t think anyone could have imagined that in a half century we would be engineering human cells and viruses to fight cancer. Over a single career, we have gone from a therapeutic arsenal of cytotoxic medicines to one that includes targeted agents such as monoclonal antibodies and “living drugs” comprised of human cells and viruses. Yet we are still in the early days of the immunotherapy revolution. There are more great things to come.