Introduction

Metal-organic frameworks (MOFs) – network solids comprising inorganic clusters connected by organic linkers into multidimensional reticular structures – have a number of physical properties which are desirable for their application as nanoparticulate drug delivery systems (DDSs)1,2,3,4,5,6,7. High molecular storage capacities, chemically addressable surfaces, and broad chemical versatility to ensure biocompatibility and enable multifunctionality have, in combination with simple size-controlled syntheses, underpinned an almost exponential growth in the volume of publications covering the applications of MOFs in the biomedicinal field8. Reproducibility of results, both chemical and biological, is essential to not only sustain the academic growth of the field, but to ensure the quickest possible pathway to clinical translation for leading candidate materials. Many of the advantageous properties of nanomaterials stem from the fact that they are not simply molecules, yet this also brings associated challenges in synthesis, characterization, and reproducibility, which some believe has hindered their real-world applicability9. With specific focus on MOFs, at present only two materials have entered human trials – RiMO-401, phase I, NCT06182579; RiMO-301, phase II: NCT0583872910 – and some have expressed opinion that translation is slower than may be expected11. In the field of drug delivery in general, reproducibility has been highlighted as a potent barrier to clinical translation12,13, and it would be unwise to assume that reproducibility problems do not also apply to MOF-based research. This perspective highlights some common issues which I believe currently hinder reproducibility in research into MOFs in nanomedicine, and offers some suggestions and resources for best practice in both the preparation of MOFs and their in vitro investigations.

Synthesis and characterisation

Clinical translation of MOFs requires reproducible syntheses with minimal batch-to-batch variation in key physical properties such as particle size, surface chemistry, and porosity. However, the capricious nature of MOF crystallisation can often result in difficulty reproducing the desired phase, never mind the same phase with broadly similar properties, in syntheses that are often low yielding. A recent inter-laboratory study of the syntheses of the Zr-porphyrin MOFs PCN-222(Zr) and PCN-224(Zr) highlighted this fact. Across ten labs globally, only one team reproduced phase pure PCN-222 from validated, previously published synthetic protocols, and no teams synthesised phase pure PCN-224, although three were able to isolate the related disordered dPCN-224 phase14. This is particularly pertinent, given that recent studies have established PCN-222(Zr) as a leading candidate for the delivery of large biomolecules as a consequence of its chemical stability and mesoporosity15. In fact, most common metal-ligand systems can result in multiple different MOF phases; polymorphism in crystalline small-molecule drugs is a significant and potentially costly problem in drug development16, so in MOF-based drug delivery, researchers face a significant burden of proof to confirm the purity of the synthesised MOF in the context of minor crystalline or amorphous impurities. For example, in addition to the archetypal Zr terephthalate MOF UiO-6617 – one of the most studied18 MOFs for drug delivery applications – there are at least four alternative phases19,20,21,22 with different secondary building units and topologies that can be isolated by minor modifications in synthetic conditions. Similarly, there are at least five phases that are commonly reported for MOFs of terephthalate with trivalent metal cations. In our own study of the Fe-terephthalate phase space, we found that similarities in the powder X-ray diffractograms of different MOFs, in concert with weaker diffraction from intrinsically flexible materials that can be isolated in different levels of structural openness, seemingly led to a number of publications where samples exhibiting diffractograms characteristic of MOF-235(Fe) were incorrectly identified as both MIL-53(Fe) and MIL-88B(Fe)23. This is again notable, as both of these Fe MOFs have been examined in the context of biological applications24,25. We have found across many studies that the use of modulated self-assembly protocols, where additives to control pH and coordinative equilibria are included in MOF syntheses, offers greater control and reproducibility of phase formation, and is highly applicable to the size-controlled syntheses of MOF nanoparticles for drug delivery work26. Nevertheless, synthetic reproducibility remains a challenge across a broad swathe of MOF research, and the application of nanomaterials to biological applications in general9.

Even with a target MOF in hand, the ability to compare one’s own results with those emerging from different laboratories, or even reproduce existing published work, is often hindered by both the variations in published synthetic techniques and a lack of reporting of key experimental data. In the context of batch-to-batch synthetic variations, it is vital to ensure a range of relevant physical parameters of MOFs are precisely characterised, because features such as particle size, morphology and surface chemistry are known to modify endocytosis pathways and efficiencies27, and also the toxicities of MOFs28. To ensure full characterization, several parameters must be examined. Chemical composition can be analysed by various elemental analysis methodologies, thermal analysis, and quantitative and/or qualitative spectroscopic techniques. The structure and crystallinity of the MOF can be determined primarily by diffraction experiments, but potentially also by X-ray or neutron scattering and X-ray absorption spectroscopy. The size and morphology of the MOF particles can be characterised in the solid-state by electron microscopy and in the colloidal form by dynamic light scattering, combined with zeta potential measurement to assess surface chemistry, in different solvents and buffers. It is necessary to compare these measurements to assess aggregation and formation of surface phenomena such as a protein corona. Gas adsorption measurements can determine porosity and so probe molecular storage capacity and, to some extent, overall crystallinity. By combining these measurements, it is possible to infer the defectivity of the synthesised MOF materials; this is vital, as a recent report has shown that differing levels of defectivity in MIL-100(Fe) can influence chemical stability, drug loading capacity, drug release kinetics, and critically in vitro toxicity and inflammatory response29. It is also vital to assess the presence of any solvents of synthesis prior to biological work, and the chemical stability of the particles, particularly as many bare MOFs have been shown to be relatively unstable in commonly used biological buffers or media30,31,32,33. High performance liquid chromatography and inductively coupled plasma mass spectrometry can quantify leaching of the organic and inorganic components, respectively, when MOFs are contacted with biological fluids.

Whilst there are clearly problems with reproducing the syntheses of certain MOFs, there is also the problem of a lack of standardisation in protocols used to synthesise and characterise MOFs. This effectively means that samples of individual MOFs will vary in key physical properties from laboratory to laboratory, making comparison of biological data difficult. To exemplify the differences in practice in the MOF literature, I have arbitrarily selected ten publications from 2023 which discuss the use of UiO-66 in biomedical applications and collated the available synthetic and characterisation data34,35,36,37,38,39,40,41,42,43. Note that this is an exercise in highlighting variations in publication practice, and not criticism of any individual study or research team. Some pertinent synthetic parameters are shown in Table 1. To summarise, none of the ten UiO-66 syntheses are the same. All involve the solvothermal reaction of ZrCl4 and terephthalic acid (TPA) in N,N-dimethylformamide (DMF), predominantly at 120 °C for 24 h but with some significant variations in time and temperature. Reaction stoichiometries most commonly follow a 1:1 metal to ligand ratio, but three differ, up to a 1:4.5 metal to ligand ratio. Reaction concentrations (defined here as the concentration of Zr in DMF used, and not including modulating solvents) vary from 8 to 169 mmol L−1, with one undefined. Four reactions use acetic acid as modulator, two use HCl (all in varying amounts relative to the Zr source), while one uses benzoic acid, one an undefined amount of formic acid, and two are unmodulated. Work-ups also vary in the solvents used to wash the as-synthesised MOF and the application of heat and/or vacuum to dry the samples.

Table 1 Comparison of synthetic parameters for ten syntheses of UiO-66 reported in 2023 for biomedical research
Full size table

These significant variations across synthetic parameters are borne out in the reported physical characterisation data, which are collated in Table 2. Qualitatively, notable differences in the levels of characterisation are evident. Two of the papers do not provide powder X-ray diffraction analysis to confirm the formation of the target phase, and in one paper the as-made UiO-66 is not characterised at all prior to loading with the anticancer drug Doxorubicin. All provide scanning and/or transmission electron microscopy (SEM and/or TEM), but with varying depths of field; one paper only shows a single UiO-66 nanoparticle by TEM. In most cases, the reader is left to assess particle sizes, which range from 50 to 250 nm, and none use image analysis to obtain a particle size distribution. Only half the studies complement this size analysis with dynamic light scattering (DLS) to determine hydrodynamic radii, with values ranging from 29 to 246 nm (two report water, and three studies do not detail the solvent used for the DLS experiments) although not all of the studies applied UiO-66 dispersed in aqueous solvent.

Table 2 Comparison of selected physical characterisation data for ten syntheses of UiO-66 reported in 2023 for biomedical research
Full size table

Accessible porosity, vital for efficient drug loading, is reported in six of the studies using N2 adsorption isotherms at 77 K. Brunauer-Emmett-Teller (BET) areas vary from 716 to 1456 m2 g−1, but only two of the six are accompanied by BET fits, which makes comparison more difficult given the known reproducibility issues that can occur when deriving BET areas from MOF adsorption isotherms44. Most provide some level of thermal analysis – thermogravimetric analysis (TGA), differential thermal analysis (DTA), or differential scanning calorimetry (DSC) – but only a few provide elemental analysis in the form of energy-dispersive X-ray spectroscopy (EDS) or X-ray photoelectron spectroscopy (XPS), which themselves only probe particle surfaces with limited penetration. Overall composition and defectivity are rarely discussed, despite the prevalence of defectivity in UiO-6645 and the potential biological influence29 of defects.

It should be noted that the syntheses here are predominantly targeted towards nanoparticle formation, driven by the requirements of the specific biological application. Different physical properties may be desirable for different applications – for example internal porosity needs not be optimised for binding cargo at particle surfaces – and so syntheses are often tailored with specific requirements in mind. The development of milder, more biocompatible protocols should also allow greater control over key physical properties46.

The diversity of synthetic procedures and key physical properties of the resulting UiO-66 samples over these ten publications illustrate the difficulties in comparing (and therefore reproducing) results and trends across different drug delivery studies. It is therefore imperative that researchers carefully characterise each batch of MOF – even if they are repeating an established protocol in the same laboratory – to ensure any variations in important physical properties are identified, and to allow any changes in biological activity to be appropriately rationalised. MOF-based DDSs are often constructed by multi-step postsynthetic processes where drugs are loaded, surface functionality is conjugated, etc. It is again best practice to carry this whole process through from one initial batch of MOF material, and to fully characterise each intermediate along the synthetic pathway. Doing so allows proper scrutiny of the physical effects of each postsynthetic modification, and also provides the most appropriate control materials to assess the biological implications of each additional functionalisation; it is not enough to rely on data produced for previously synthesised (or reported) batches of MOF. Reproducibility would certainly be assisted with some field-specific standards on making experimental data open access. Other than community-driven initiatives such as the Adsorption Information File (AIF) for porosity data47, archiving raw data occurs in a piecemeal fashion, usually driven by research funder regulations, when a standardised approach would be hugely beneficial for the field. Indeed, others have opined that a lack of availability of raw data hinders reproducibility of scientific research in general48.

In vitro experiments

Beyond the synthetic aspects described above, it is logical to predict that research into drug delivery from MOFs will experience the same reproducibility issues that are prevalent across biomedical research in general. These include, but are not limited to, the complexity of DDSs hindering repeat manufacture12, so-called “P-hacking” where data analysis is misused to generate statistical false positives49, contamination and misidentification of cell lines50, the inapplicability of certain animal models13, and even research misconduct; a large body of fabricated papers on MOFs for biomedical applications has recently been uncovered51.

The in vitro biocompatibility of MOFs is regularly reported in the literature, but is assessed against a wide range of predominantly immortalised cell lines using many different assays. Comparison of data across labs, and thus reproducibility of results, is again made difficult by these variables28. Choice of cell line is important; immortalised cell lines (both healthy and diseased) are widely used due to their robustness, yet primary cell lines derived from patients may be more representative of in vivo conditions, although come with a limited lifespan. Colorimetric and fluorescent proliferation assays52 are routinely employed to assess cytotoxicity, whilst flow cytometry and real-time cell analysis methods are also reported, but less frequently. Our own experience is that colorimetric assays, such as the ubiquitous 3-(4,5-dimethylthiazol-2-yl)−2,5-diphenyl tetrazolium bromide (MTT) and 3-(4,5-dimethylthiazol-2-yl)−5-(3-carboxymethoxyphenyl)−2-(4-sulfophenyl)−2H-tetrazolium (MTS) assays, can produce false positives if MOFs aggregate and sediment during incubation, issues that do not affect fluorescent assays such as alamarBlue. These types of assay probe cellular metabolism rather than viability, so using them to determine biocompatibility can provide misleading results. We have, therefore, found real-time cell analysis to be a more sensitive protocol to detect disturbances in cell growth as opposed to these end-point proliferation assays, although their simplicity makes them a valuable initial tool53.

There are a wide range of factors which influence the toxicity of MOF nanoparticles towards particular systems – the reader is directed to a comprehensive recent overview for further details28 – so it is not appropriate to rely on previous declarations of biocompatibility, or inferring this from the toxicity profiles of the constituent metals11, when carrying out a new study. It is best practice to ensure studies are designed with appropriate control experiments specific to the batch of MOF being used and the particular cell culture; it is well established that cells of different passage number can behave differently54, meaning comparison (and thus reproducibility) of results across commonly used cell lines is hindered as this parameter is not commonly reported. The reader is directed to comprehensive overviews of best practice in cell culture for more information55,56.

Publication of raw in vitro assay data is as important, if not more important, than the availability of synthetic data. Comparison of data within an individual publication is carried out by the study’s authors, and should be accompanied by appropriate statistical analysis to determine the level of significance of any observed effects across a series of biological and technical replicates. Unless full raw data are published, however, it is not possible for other researchers to make statistical comparisons with their own data (assuming appropriate controls have been performed). Similarly, when statistical analysis is not provided in a publication, external scrutiny is not possible without full archiving or reporting of the raw data. Unfortunately, most publications present in vitro results in graphical form and deposition of raw data is again not the standard in the field. Development and adoption of a standard format data deposition file analogous to the AIF would be hugely beneficial in this regard, and also provide an alternative option for the publication of negative results or failed attempts to reproduce published data, another suggested strategy to drive the research and save resources12. Whilst this perspective has focussed on in vitro work, there are significant numbers of in vivo studies involving MOFs emerging in the literature; similar concerns around study design and reproducibility exist in animal research13,57.

Outlook

With the first MOFs now undergoing human trials, it is clear that there is huge potential for their application in biomedicine, and it is incumbent on scientists in the field to ensure that reproducibility and transparency underpin their research programs, from study design, through research protocols and ultimately in publication and communication practices. The explosive growth of research in the area has contributed to a lack of field-specific standardisation of key synthetic and analytical parameters – what constitutes a biocompatible MOF for example – that would enable greater comparison between research teams. Similarly, I would reiterate previous pleas for publications to be accompanied by the archiving of raw data, but add that more detailed descriptions of experimental processes and more comprehensive characterisation of synthesised MOF materials are also required. In some ways, MOF researchers can take heart that the issues discussed in this perspective pervade nanomedicine in general; we can learn from (and contribute to) the ongoing debate as to best practices and community standards58 in a mutually constructive manner. Truly interdisciplinary research teams ensure a range of views, skills, and experience are intrinsic to any research study, providing beneficial added value and maintaining relevancy, and are strongly recommended in this field which straddles both materials chemistry and biology. By embracing the principles of trusted research and innovation, it should be possible to drive more materials towards clinical translation and realise the extraordinary potential of MOFs in medicine.