Comparative Analysis of HtrA3 and NGAL as Viable Biomarkers for Pre-eclampsia

Pre-eclampsia is a pregnancy-associated condition, which is characterized by the onset of hypertension and proteinuria. It is one of the leading causes of maternal and neonatal mortality and this affliction has been recorded in around 8% of all pregnancies in the world. In addition to this, the etiopathology of this condition is very less understood and the resources available to diagnose and treat it are limited. Prior studies suggest more than a hundred possible diagnostic biomarkers that could be used to detect this disease early on. However, most of them are not feasible due to several reasons including stability, cost, safety, etc. Here two biomarkers HtrA3 (high-temperature requirement A3) and NGAL (Neutrophil Gelatinase Associated Lipocalin) are selected for the detection of pre-eclampsia, and we compare their efficacy in the detection of pre-eclampsia based on their specificity, ease of use, speed, stage of detection and source (invasiveness). We found that these two biomarkers are efficient under some parameters, and inefficient under others. The scoring system used in the current study suggests that NGAL is a superior biomarker. The results of this study help to develop a stronger understanding of both these biomarkers in the short and long term to classify the biomarkers more efficiently and understand the complicated pathologies of pre-eclampsia.

Pre-eclampsia is a pregnancy complication that has been classified as a hypertensive disorder of pregnancy. It has been a major cause of maternal and fetal morbidity and mortality in developing countries including Latin America, Africa, and India. Pre-eclampsia is responsible for ~60,000 annual maternal deaths worldwide and ~8.3% maternal mortality rate in India [1]. This is a multisystem disorder that collectively affects the renal, hepatic, neurological, and cardiopulmonary systems [2]. Manifestation of its symptoms begins post 20 weeks of gestation which include hypertension, proteinuria, HELLP Syndrome (Hemolysis, Elevated Liver enzymes, and Low Platelets), water retention and swelling due to edema, nausea, upper abdominal pain, and migraines [3]. There is a lack of clear etiopathology of this disease and some studies attribute it to the improper remodeling of the uteroplacental arteries in the early stages of pregnancy [4]. This leads to poor placental perfusion, impaired placental development, and its function. Eventually, it causes a surge in the amount of placental and angiogenic factors in maternal circulation- which triggers endothelial dysfunction [5]. Decreased placental perfusion causes a major risk to the fetus stems which in turn leads to the decreased blood supply, oxygen, and nutrients that are required for fetal growth and development [6]. This disorder has further been classified into mild or severe based on the factors such as blood pressure (BP), proteinuria levels, and platelet count. “Mild pre-eclampsia” is categorized with BP ≥140/90 mmHg, but less than 160/110 mmHg with proteinuria ≥300 mg/24 h while “Severe pre-eclampsia” is classified with BP ≥160/110 mmHg with urinary protein excretion of ≥2.0 g/24 h or any of these, oliguria or <400 ml urine/24 h, visual disturbances, serum creatinine ≥1.2 mg/dl, platelets less than 100,000/mm3, and hemolysis. [7]. Here we aim to compare high-temperature requirement A3 (HtrA3; henceforth) and Neutrophil Gelatinase Associated Lipocalin (NGAL; henceforth) as prospective biomarkers for pre-eclampsia. This was achieved by:

a. Setting up a specific set of parameters.

b. Establishing a scoring system to compare them with said parameters.

Existing treatment

Diagnosis and detection of pre-eclampsia in its early stage has been a major challenge of modern obstetrics. While its diagnosis, doctors take various risk factors into account such as the family history of pre-eclampsia, chronic hypertension, first pregnancy, race, maternal age, obesity and multiple pregnancies (this condition is exacerbated in women carrying twins or triplets [8]. In countries across America and Europe, a low dose of Aspirin is prescribed for patients that are exhibiting certain risk factors. A dosage of ~ 81 mg of Aspirin beginning after 12 weeks of pregnancy is thus prescribed for such patients. Methyldopa is a commonly used adrenergic agonist drug that manages hypertension [5]. Alpha methyldopa (FDA category B drug) has been prescribed in case of mild pre-eclampsia and is safe for both mother and the fetus while in case of severe pre-eclampsia, calcium channel blocker with methyldopa is prescribed [5].

Biomarkers

Biomarkers are the quantifiable indicators of biological processes either the products or side products of the biological reactions. According to the WHO, a biomarker can be defined as “any measurement reflecting an interaction between a biological system and a potential hazard, which may be a chemical, physical, or biological entity. The measured response may be functional and physiological, biochemical at the cellular level, or molecular interaction” [9]. The two main factors required to assess the quality of a biomarker are relevance and validity. Relevance refers to the clinically relevant and accurate results that a biomarker can provide whereas validity refers to the effectiveness or the utility used for evaluation. An optimal biomarker must comprise the disease-specific sensitivity and ubiquity for a diseased population with accurate results across a wide range of patients. Moreover, good biomarkers are capable of reducing the time and cost of detection for a particular condition [10].

NGAL

NGAL, commonly known as Lipocalin-2 is a member of the adipokines superfamily. These molecules are involved in pathophysiological processes such as metabolic homeostasis, apoptosis of hematopoietic cells, transport of fatty cells, neuroendocrine functions, and blood pressure [11]. In recent studies, NGAL has also been linked to the pathogenesis of metabolic disorders due to its effects on inflammation [11]. Structurally, these proteins consist of a single disulfide bridged polypeptide chain of 178 amino acids and it has emerged as a novel biomarker for kidney injury and ischemic perfusion injury [5].

NGAL is also expressed in immune cells and human tissues, including the uterus, kidney, breast, and colon; and it shows elevated levels in epithelia that have been damaged due to inflammation [11]. Generally, NGAL binds to the matrix metalloproteinase-9 (MMP-9) protein to protect itself from auto degradation [12]. Overexpression of NGAL leads to overexpression in MMP-9 levels, which in turn leads to increased angiogenesis and tumor cell proliferation. Hence, serum and urine NGAL levels can be correlated to residual renal function [13]. NGAL is an acute phase protein- a class of proteins whose concentrations in blood plasma either increase or decrease in response to inflammation [5]. NGAL levels are usually low in biological fluids, yet they get upregulated in inflammatory states. Studies show that NGAL levels are particularly elevated at the end of the second trimester in women who subsequently developed pre-eclampsia [5]. This protein has become increasingly relevant as a potential clinical biomarker in recent years on account of the fact that its levels in biological fluids are generally low, but show an upregulation in cases of inflammation [5]. This strongly indicates the use of NGAL as a clinical biomarker.

Detection of NGAL

The most commonly used method for detecting and analyzing NGAL levels is the Enzyme-Linked Immunosorbent Assay (ELISA) [5]. The completed methodology is described elsewhere [5]; briefly, for the detection of NGAL levels, samples of blood and first-morning mid-stream urine samples of pregnant women were collected (Figure 1). These samples are then immediately centrifuged at 15 min at 4 ℃ to remove the debris. Supernatants are then separated and stored in 1.5 mL aliquots along with protease inhibitors in order to prevent proteolysis at -80 ℃. Thereafter, NGAL levels are detected by ELISA and the results are measured at a particular wavelength (450 nm +/- 10 nm) and the results are thus compared with a standard curve. The inter and intra-assay coefficients are subsequently noted to identify any inconsistencies or possible errors in ELISA results.

HtrA3

Mammalian HtrA3 is a serine protease of the HtrA family, which is important for placental development or cancer progression and there are 4 recognized HtrA homologs, 3 of which have been extensively studied [14]. HtrA4 is the most recently discovered and extensive studies are required for its holistic understanding. HtrA3 has been associated with trophoblast invasion which usually occurs within the first trimester of pregnancy [15]. Taking this into account, it suggests that this biomarker can be detected early on. As there are no high throughput means of extracting HtrA3 from the human serum, which is almost hard to detect. Hence, the conventional method of using ELISA is not recommended [16].

Detection of HtrA3

Studies have shown that HtrA3 has two isoforms - the long isoform (HtrA3-L) and the short isoform (HtrA3-S) [17]. These isoforms are nearly identical, except for the fact that HtrA3-S lacks a C-terminal PDZ (postsynaptic density protein) domain. Generally, Western blot analysis is used to analyze HTRA3 levels. The completed methodology is described elsewhere [17] briefly a set of HTRA3 mouse monoclonal antibodies (mAbs) are used to recognize and differentiate the two isoforms [10]. A technique known as AlphaLISAs or Amplified luminescent proximity homogeneous assays-linked immunosorbent assays are used to detect the HTRA3 isoforms in picomolar levels in serum. This technique is an iteration of ELISA, it is a bead-based luminescent amplification assay that offers greater sensitivity as well as a wider range of smaller-sized sample detection as compared to traditional ELISA [18].

Selection of biomarkers

In this study, we aim to perform a novel comparative analysis of two biomarkers that are associated with pre-eclampsia and hypertensive gestational stress, which do not comprise the picking of arbitrary biomarkers that may have a passing relationship with this malady. However, the selection is done by considering the following few factors:

1. The two biomarkers must have a significant and specific role in pre-eclampsia.

2. They must have (to a great extent) comparable features, i.e. one biomarker must not significantly be better than the other.

3. There must not be a significant similarity between the two biomarkers.

Considering these factors, we referred to the list of potential biomarkers for pre-eclampsia. Acestor, et al. [19] have determined 135 recognized potential biomarkers; among these 77 biomarkers were determined to be unstable or incompatible with gestational disorders, while the remaining 58 biomarkers were assessed for similarity (Figure 1) [19].

One of the crucial elements for comparison was the sources of samples. Around 9 of the acceptable biomarkers were urinary-based, while rest were the serum based. Since the general constituents of the urinary and serum-based samples are varying; it is essential to pick the biomarkers from the same source. Serum-based samples have proven to be a more reliable source, despite being more invasive in nature. The ideal biomarker would be fast, and reliable, along with non-invasive. Considering these factors, the biomarkers that have been chosen are High Temperature Resistant Serine Protease A3 (HtrA3), and Neutrophil Gelatinase Associated Lipocalin-2 (NGAL). It was observed that several of their parameters were of comparable values (Figure 1).

Selection of parameters

It is important to note that the comparison of one specific parameter could provide the perception of one biomarker being superior to the other by default. Here we provide an unconstrained view of all comparable parameters to develop a comprehensive overview of both biomarkers. The four highly prominent comparative parameters are as follows:

1. Cost: It is important for the detection of biomarkers to be economically viable. This revolves partly around the process involved, and partly on the required reagents for the same. One can determine the cost by economic analysis and optimization.

2. Specificity and sensitivity: These are the arguably most important factors when considering the choice of biomarkers. There are some effective biomarkers in detecting hypertensive disorders, like inositol phosphoglycan type P, but are not specific to pre-eclampsia, hence providing inconclusive results. At the same time, there are conventional biomarkers like soluble ST2 (Suppression of Tumorigenicity 2), which are mildly sensitive to hypertensive disorders. The two factors are put in adjacency as they both can be measured using the Area Under Curve (AUC) for the Receiver Operating Characteristic (ROC) curve.

3. Time of detection: During pregnancy, this refers to the point at which the biomarker can be tested most efficiently. This is a parameter specific to gestational disorders and is important on account of the fact that early detection of pre-eclampsia is preferred in order to offer its effective treatment.

4. Simplicity of process: While deciding on a biomarker, one must keep in mind the simplicity of its detection. Some biomarkers require complicated procedures and expensive means, while others are much faster-paced, and can be done in bulk if necessary. Therefore, we are considering this as an important factor for the current study.

Scoring

In order to achieve a comprehensive evaluation of the biomarkers, a clear definition of superior biomarkers within the constraint of a parameter would be helpful. Here a binary system has been developed by assigning a lower value - 0, and an upper value -1. Upon comparison, the quantifiable data were verified, and the better of the two data was assigned a value of 1. This was done in order to provide a comprehensive evaluation of both quantitative and qualitative parameters. Upon comparison, the quantifiable data were verified, and the preferable of the two data was assigned a value of 1. The other data was given a relative value from 0 to 1. Moreover, the qualitative data were also associated with the quantifiable data, and the simplicity of the process was associated with the time it took to carry out the detection procedure. The simplicity score for NGAL was given a score of 1 because of the fact that ELISA for Lipocalin-2 is ~15 minutes long, while that of AlphaLISA for HtrA3 is 2 - 3 hours long. Comparatively, the score of HtRA3 was computed to ~0.1.

Essentially, the score is given by:

$\text{Score (Positive correlation) = }\frac{\text{Value of lesser effective biomarker}}{\text{Value of more effective biomarker}}\text{ × 1}\text{.0}$

$\text{Score (Negative correlation) = }\frac{\text{Value of more effective biomarker}}{\text{Value of lesser effective biomarker}}\text{ × 1}\text{.0}$

Similarly, values for all other parameters were calculated.

To study the biomarkers, we first established a list of potential biomarkers [19], which includes 135 biomarkers. The current study further clarifies the stability of these biomarkers. Among the remaining biomarkers, we selected two serum-based biomarkers, NGAL and HtrA3 to test the various parameters. A survey of these two biomarkers separately provides us with enough data to decide and evaluate the values for other parameters. The selected parameters were the cost, specificity, sensitivity, time of detection, and simplicity, that we obtained from the literature survey (Figure 1). Factors like cost and time of detection are inversely correlated with preference, while specificity, sensitivity, and simplicity were given a positive correlation.

ROC

Receiver Operating Curve (ROC) is a graphical representation of the diagnostic capability of a biomarker. Specificity and sensitivity (for ROC) are the two parameters through which, we can set different thresholds to interpret the trade-offs between true positive rate (sensitivity) and false positive rate (1-specificity). The false positive rate is plotted along the abscissa while the true positive rate is plotted along the ordinate [20]. A random classifier that depicts a biomarker that can neither have true nor false positives is plotted along the diagonal. Any curve that extends above the diagonal can be considered a functional biomarker whereas the curve that extends below it is an ineffective one [2,21]. The area Under the Curve (AUC; henceforth) is measured to understand the efficiency of the biomarker; an AUC of 0.5 or below implies that the biomarker is unable to distinguish between affected and unaffected patients by the disease. Similarly, an AUC of 0.7- 0.8 and >0.9 is considered acceptable and an excellent AUC score respectively. The ROC was used to measure the sensitivity and specificity of threshold values for both biomarkers. The threshold value was set based on the available literature. Hence, no pre-processing steps on the data set were required. The AUC was then subsequently calculated as the simple area under the ROC curve, thus representing the overall performance of the biomarker.

Establishment of priority

The cumulative score for a biomarker ponders the relative importance of each parameter based on the situation. For instance, large-scale testing for pre-eclampsia in rural areas would give significant importance to the cost and simplicity of the process. Factors of importance such as a, b, c, and d are considered for cost, simplicity, specificity, time of detection during gestation, and simplicity respectively. These factors help to generate the formula for a cumulative score as follows:

Cumulative score = a (Cost score) + b (AUC score) + c (Time of detection score) + d (Simplicity score).

For the current study, we assume an equal weightage to all the parameters, i.e. a, b, c, d = 1.

Based on the literature survey, the following data are scored as discussed before:

Cost

This is the data as of 2021 for the required diagnostic set-up of NGAL and HtrA3 (Table 1). The AlphaPlate-384 was used for the HtrA3 detection set-up, while Monkey ELISA Kit-LS-F38530 was used to detect NGAL. The total optimal cost was determined based on the market price, and the minimum required units for a diagnostic setup.

HtrA3 was recognized with a lower cost of Rs. 7250 and was given a score of 1 while the cost of Rs. 17886 was designated for NGAL (Table 1). Relative scoring for negative correlation is given by the inverse ratio (that is, the cost is inverse) and is inversely proportional to the score awarded, as seen below:

Score (Cost of NGAL) = (7250/17886) × 1 = 0.41

Specificity and sensitivity

These two factors are calculated with the use of a Receiver Operating Curve (ROC). Instead of using the individual values of specificity and sensitivity, the area under the ROC is used as the deciding criteria. The AUC for HtrA3 was found to be 0.79, while that for NGAL was found to be 0.95 (Figure 2; Table 1). Hence, a positive correlation is reported and NGAL is found to be the superior biomarker (so marked its score as 1), while the relative score for HtrA3 was found to be,

Score (AUC of HtrA3) = (0.79/0.95) × 1 = 0.83

Time of detection

Based on a literature survey, the optimal time for detection for HtrA3 and NGAL during pregnancy is found to be around 15 - 17 and 19 - 20 weeks into gestation respectively. In pre-eclampsia, earlier detection is preferred in order to begin the initial treatment early on. Once again, because of the negative correlation, HtrA3 is the preferred biomarker, giving it a score of 1. The relative score was found to be 0.83 for NGAL based on the above formula (Table 1).

Simplicity

Here we compared various steps involved in the diagnosis of NGAL and HtrA3. As discussed earlier, the procedure for the detection of the two biomarkers was carefully studied, and the average time of diagnostics was estimated. The time of diagnostics for NGAL was found to be around 15 to 20 minutes. AlphaLISA is a significantly more time-consuming process and estimated to be around 2 to 3 hours long. There is once again a negative correlation between this parameter and the score. The time (in minutes) was used in relation to the score. HtrA3 detection is approximately 150 minutes long. Since NGAL is faster, it is scored 1 while the relative score of HtrA3 was found to be 0.1 (Table 1).

Summation (NGAL) = 1 (0.41) + 1 (1.00) + 1 (0.83) +1 (1.00) = 3.24.

Summation (HtrA3) = 1 (1.00) + 1 (0.83) + 1 (1.00) + 1 (0.1) = 2.93.

Our study and the designated scoring system suggest that NGAL is a better biomarker than HtrA3 on account of its higher score, which was calculated after considering various parameters such as cost, specificity-sensitivity, simplicity of the process, and the period of pregnancy during which the condition was detected. Under the circumstances described and the parameters chosen, this would indicate that NGAL is going to be preferable in the early detection of pre-eclampsia. Taking an optimum, equal view of all parameters, the detection of NGAL would provide a better early diagnosis of pre-eclampsia. However, it is crucial to note that the parameters and their relevance could vary strongly in different circumstances. For instance, lower economy countries could place a significantly higher importance to cost, which could make them provide a higher weightage to the scoring of cost. Through extension, this process of scoring could be performed with other biomarker detection in pre-eclampsia, in other maladies, or even evaluating different choices of drugs for study.

Specific to our study, we also considered the individual parameters while selecting the biomarkers. For instance, while performing large-scale testing for pre-eclampsia; one would give more importance to the speed and cost of detection, while less importance to the specificity/sensitivity. On the other hand, HtrA3 being less specific and sensitive is much more economical and is capable of delivering the results very early during pregnancy. We report NGAL to be more effective in terms of simplicity and the AUC score (generated from its ROC- a curve that indicated greater specificity and sensitivity of a biomarker).

Further work

The current study shows that there is a significant lack of funding for pre-eclampsia and hypertension studies. Millions of dollars are spent each year on neonatal treatment, which can be avoided while taking prior action. Further studies into the various biomarkers that we could not include in this study might significantly enhance our understanding of effective diagnosis. Furthermore, continued studies into pre-eclampsia pathology could help us understand the various biochemical pathways involved and provide us with enough data to completely and economically avoid this malady in the future.

PY acknowledges the SASTRA Deemed to be University for the infrastructural facilities.

Funding: Science and Engineering Research Board, DST, Government of India CRG/2019/003184 (PY).

Author contributions: Conceptualization: SNC, SSS, PY; Methodology: SNC, SSS, PY; Investigation: SNC, SSS, PY; Visualization: SNC, SSS, PY; Supervision: PY; Writing- original draft: SNC, SSS, PY; Writing- review & editing: SNC, SSS, PY.

1. Wang T, Zhou R, Gao L, Wang Y, Song C, Gong Y, Jia J, Xiong W, Dai L, Zhang L, Hu H. Elevation of urinary adipsin in preeclampsia: correlation with urine protein concentration and the potential use for a rapid diagnostic test. Hypertension. 2014 Oct;64(4):846-51. doi: 10.1161/HYPERTENSIONAHA.113.02688. Epub 2014 Jun 23. PMID: 24958499.
2. Xiao J, Niu J, Ye X, Yu Q, Gu Y. Combined biomarkers evaluation for diagnosing kidney injury in preeclampsia. Hypertens Pregnancy. 2013 Nov;32(4):439-49. doi: 10.3109/10641955.2013.827203. Epub 2013 Aug 19. PMID: 23957217.
3. Dekker GA, Sibai BM. Etiology and pathogenesis of preeclampsia: current concepts. Am J Obstet Gynecol. 1998 Nov;179(5):1359-75. doi: 10.1016/s0002-9378(98)70160-7. PMID: 9822529.
4. Sachan R, Patel M, Gaurav A, Gangwar R, Sachan P. Correlation of serum neutrophil gelatinase associated lipocalin with disease severity in hypertensive disorders of pregnancy. Adv Biomed Res. 2014 Nov 29;3:223. doi: 10.4103/2277-9175.145690. PMID: 25538909; PMCID: PMC4260273.
5. Krishna U, Bhalerao S. Placental insufficiency and fetal growth restriction. J Obstet Gynaecol India. 2011 Oct;61(5):505-11. doi: 10.1007/s13224-011-0092-x. Epub 2011 Nov 17. PMID: 23024517; PMCID: PMC3257343.
6. Upadya M, Rao ST. Hypertensive disorders in pregnancy. Indian J Anaesth. 2018 Sep;62(9):675-681. doi: 10.4103/ija.IJA_475_18. PMID: 30237592; PMCID: PMC6144552.
7. Kuo HH, Yang JM, Wang KG. Preeclampsia in multiple pregnancy. Zhonghua Yi Xue Za Zhi (Taipei). 1995 May;55(5):392-6. PMID: 7641125.
8. Khalid A, Byrne BM. Aspirin in The Prevention of Pre-Eclampsia: Where Are We Now? Ir Med J. 2018 Mar 14;111(3):704. PMID: 30376222.
9. Wang Y, Li Y, Hyett J, da Silva Costa F, Nie G. HtrA3 Isoform-Specific ELISAs for Early Detection of Preeclampsia. SLAS Discov. 2018 Dec;23(10):1092-1099. doi: 10.1177/1087057116682425. Epub 2016 Dec 8. PMID: 27932697.
10. Abella V, Scotece M, Conde J, Gómez R, Lois A, Pino J, Gómez-Reino JJ, Lago F, Mobasheri A, Gualillo O. The potential of lipocalin-2/NGAL as biomarker for inflammatory and metabolic diseases. Biomarkers. 2015;20(8):565-71. doi: 10.3109/1354750X.2015.1123354. Epub 2015 Dec 15. PMID: 26671823; PMCID: PMC4819811.
11. Bouchet S, Bauvois B. Neutrophil Gelatinase-Associated Lipocalin (NGAL), Pro-Matrix Metalloproteinase-9 (pro-MMP-9) and Their Complex Pro-MMP-9/NGAL in Leukaemias. Cancers (Basel). 2014 Apr 4;6(2):796-812. doi: 10.3390/cancers6020796. PMID: 24713998; PMCID: PMC4074804.
12. D'Anna R, Baviera G, Giordano D, Todarello G, Russo S, Recupero S, Bolignano D, Corrado F. Neutrophil gelatinase-associated lipocalin serum evaluation through normal pregnancy and in pregnancies complicated by preeclampsia. Acta Obstet Gynecol Scand. 2010;89(2):275-8. doi: 10.3109/00016340903443676. PMID: 19961280.
13. Glaza P, Osipiuk J, Wenta T, Zurawa-Janicka D, Jarzab M, Lesner A, Banecki B, Skorko-Glonek J, Joachimiak A, Lipinska B. Structural and Functional Analysis of Human HtrA3 Protease and Its Subdomains. PLoS One. 2015 Jun 25;10(6):e0131142. doi: 10.1371/journal.pone.0131142. PMID: 26110759; PMCID: PMC4481513.
14. Wang LJ, Cheong ML, Lee YS, Lee MT, Chen H. High-temperature requirement protein A4 (HtrA4) suppresses the fusogenic activity of syncytin-1 and promotes trophoblast invasion. Mol Cell Biol. 2012 Sep;32(18):3707-17. doi: 10.1128/MCB.00223-12. Epub 2012 Jul 9. PMID: 22778138; PMCID: PMC3430202.
15. Teoh SSY, Wang Y, Li Y, Leemaqz SY, Dekker GA, Roberts CT, Nie G. Low Serum Levels of HtrA3 at 15 Weeks of Gestation Are Associated with Late-Onset Preeclampsia Development and Small for Gestational Age Birth. Fetal Diagn Ther. 2019;46(6):392-401. doi: 10.1159/000497144. Epub 2019 Apr 23. PMID: 31013509.
16. Dynon K, Heng S, Puryer M, Li Y, Walton K, Endo Y, Nie G. HtrA3 as an early marker for preeclampsia: specific monoclonal antibodies and sensitive high-throughput assays for serum screening. PLoS One. 2012;7(9):e45956. doi: 10.1371/journal.pone.0045956. Epub 2012 Sep 25. PMID: 23049902; PMCID: PMC3457993.
17. Sakamoto S, Putalun W, Vimolmangkang S, Phoolcharoen W, Shoyama Y, Tanaka H, Morimoto S. Enzyme-linked immunosorbent assay for the quantitative/qualitative analysis of plant secondary metabolites. J Nat Med. 2018 Jan;72(1):32-42. doi: 10.1007/s11418-017-1144-z. Epub 2017 Nov 21. Erratum in: J Nat Med. 2018 Jan 5;: PMID: 29164507; PMCID: PMC5775980.
18. Acestor N, Goett J, Lee A, Herrick TM, Engelbrecht SM, Harner-Jay CM, Howell BJ, Weigl BH. Towards biomarker-based tests that can facilitate decisions about prevention and management of preeclampsia in low-resource settings. Clin Chem Lab Med. 2016 Jan;54(1):17-27. doi: 10.1515/cclm-2015-0069. PMID: 25992513.
19. Søreide K. Receiver-operating characteristic curve analysis in diagnostic, prognostic and predictive biomarker research. J Clin Pathol. 2009 Jan;62(1):1-5. doi: 10.1136/jcp.2008.061010. Epub 2008 Sep 25. PMID: 18818262.
20. Hajian-Tilaki K. Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation. Caspian J Intern Med. 2013 Spring;4(2):627-35. PMID: 24009950; PMCID: PMC3755824.
21. El-Shourbagy S, Amira A, Desouky E. Abo Amo. Predictive value of Lipocalin 2 and hyperuricemia on maternal and fetal outcome in pre-eclampsia. Evidence Based Women's Health Journal. 2017; 7:131-140.