Journal of Community Medicine & Health Education
Open Access

Clinical trials of vaccines are divided into stages I, II, Ⅲ and IV. The Phase I trial is mainly safety and may be immunogenicity. Phase II trials are further tests on human safety and immunogenicity, and phase Ⅲ studies are often a field evaluation of direct protection effects and further accumulate safety data. Among them, the phase I, II and Ⅲ clinical trials should be completed before the official marketing, mainly used to study the protective effects and risks of vaccines, and to provide reliable data support for the listed review.

At present, phase Ⅲ trials usually divide a large number of people into control groups and vaccination groups to obtain the incidence of the control group and vaccination group, and then calculate the protection rate as the protection effect of vaccination [1]. Eligibility criteria are the current evaluation indicators of vaccine protection effect of 50% or greater.

The phase Ⅲ vaccine trial is a field evaluation. According to Wang Ruyun analysis of the protection rate of phase Ⅲ trials of COVID-19 vaccine, local protective measures (public health measures) have a great impact on the current vaccine protection rate [2]. The current vaccine protection rate is not suitable for evaluation of COVID-19 vaccines. The definition of comprehensive protection rate was given and its feasibility as an evaluation indicator of vaccine protection effect was studied.

Based on the study of phase Ⅲ trials of COVID-19 vaccines, we further studies on the impact of public health measures on phase Ⅲ trials of general vaccines. The magnification of incidence bias to existing vaccine protection bias was derived. This magnification is directly controlled by the public health protection rate, the greater the public health protection rate, the greater the magnification times. It reveals the internal mechanism that the current vaccine protection rate is not suitable for the evaluation of vaccines. Data from existing phase Ⅲ clinical trials of COVID-19 vaccines also strongly suggest that the public health protection rates of current vaccines with high protection rates in phase Ⅲ trials are generally low; and vaccines tested in areas with high rates of public health protection often have low rates of current vaccine protection. The feasibility of comprehensive protection rate as evaluation indicator of vaccine protection effect was obtained by case analysis. Then, an equivalent relationship with the existing eligibility criteria for test vaccines is established by using the comprehensive protection rate with comparability and computational stability. It is proved that with the increase of public health protection rate, the eligibility criteria of phase Ⅲ clinical trial of vaccine will lead to the difficulty of vaccine passing the requirement of conformity. It is therefore recommended that the existing eligibility criteria for testing vaccines be abolished. A method to optimize the selection of vaccines was proposed by establishing a test standard for the significant protective effect of vaccines and combining with the comprehensive protective rate.

Vaccine related protection rate and eligibility criteria

Introduction of current vaccine protection rates: If the incidence of the control group and the vaccination group are r_c, r_v respectively, then the current calculation formula of vaccine protection rate is

Protection rate of public health measures: However, this protection rate in formula (1) does not take into account whether the volunteers were exposed to public health measures. Public health measures generally refer to wearing masks, washing hands frequently and keeping a safe distance. However, when comparing vaccine protection rates between different countries in the world, public health measures should also include objective factors such as population density, climate and natural environment that have protective effects on human beings.

Therefore, it is an important issue to study whether the protection rate of phase Ⅲ vaccine trials will be different in different public health environments, that is, whether public health measures will have an impact on the current vaccine protection rate.

In order to facilitate the study, the definition, calculation formula and calculation method of protection rate of public health measures are given according to formula (1).

Assuming that the local incidence rate before and after the adoption of public health measures is r_n and r_p, then the formula for calculating the protection rate of public health measures is

Definition of vaccine protection rates before and after the introduction of public health measures and the Comprehensive protective rate: If phase Ⅲ vaccine trials are conducted in an area where no public health measures are taken and the number of cases in the control group and the vaccination group are respectively marked as, m_c/n and m_v/n, then the incidence rates in the control group and the vaccination group are respectively.

Here m is the number of people in each group.

The corresponding formula for vaccine protection rate b_v/n in areas where no public health measures have been taken is

If phase Ⅲ vaccine trials are conducted in areas with public health measures and the number of cases in the control group and the vaccination group are denoted as m_c/p and m_v/p, then the incidence rates in the control group and the vaccination group are respectively.

The corresponding formula for vaccine protection rate b_v/p in areas where public health measures have been taken is

Further, assuming that the entire population is vaccinated in the area where the public health measures have been implemented, then the comprehensive protection rate b_v+p provided by the public health measures and the vaccine to the local population is calculated by the formula

In phase Ⅲ trials with large vaccine samples,

According to equations (2), (6), (7) and (8), it can be derived

Eligibility criteria for testing vaccines: The current eligibility criteria by the World Health Organization for vaccine protection is

That is, in areas where public health measures are not and have been adopted, they need to be met

The requirement b_v/n≥50% is, in fact, a special case of b_v/p≥50% in the protection rate of public health measures b_p= 0 . So the two conditions in formula (11) can be combined into b_v/p≥50%

It can be seen from Formula (9) that the qualified the current protection rate b_v/p≥50% and comprehensive protection rate b_v+p need to be met

Confirm an equivalent relationship.

The current vaccine protection rate is not suitable for evaluating the effectiveness of vaccine protection

It can be derived from equations (3) and (6)

Here Δm_v/p, Δm_c/p respectively represent the statistical bias of m_v/p, m_c/p. During the epidemic development stage, the number of cases in the phase Ⅲ trial area increased exponentially, and both the vaccination group and the control group were exposed to a sharp increase in external sources of infection, resulting Δm_v/p>0 and Δm_c/p>0 in a high probability.

As can be seen from Equation (13), as long as

Δb_v/p<0 Will appear, that is, will reduce the value of the evaluation indicators currently adopted. As can be seen from the factor (1-b_v/p) at the right end of Equation (14), when the value b_v/p is larger, the inequality is more likely to be established. That is, the better the protection effect, the easier it is to reduce the value due to the statistical bias Δm_v/p in the vaccine group, which is a very important result and needs to attract attention.

According to Equation (13), we have

This Δr_c/p represents the bias in the incidence of the control group. Since b_p, r_n and m are respectively the protection rate of public health measures in the region where the phase Ⅲ vaccine trial was conducted, the incidence rate without public health measures, and the number of people in each group in the phase Ⅲ trial, they can be set as invariants. Under the assumption that formula (8) is true, it can be derived in combination with Formula (2)

It can be seen from Formula (15) that if there is statistical bias in the number of patients in phase Ⅲ trial, its impact on the current vaccine protection rate will be magnified by factors is different between trial areas. It can be seen that the better the public health measures, the greater the amplification of statistical bias. When Table 1 shows the magnification of the statistical bias of the number of cases corresponding to different public health protection rates. As a result, b_v/p is not appropriate to evaluate the protective effect of vaccines (Table 1).

Table 1: The magnification of the statistical bias of the number of cases corresponding to different public health protection rates.

（%）	10	20	30	40	50	60	70	80	90
	1.11	1.25	1.43	1.67	2	2.5	3.33	5	10

b_v/p≥50 % Is not appropriate to be used as eligibility criteria of phase Ⅲ clinical trial of vaccine

We know that there is no place without public health measures, but the intensity of public health measures varies from place to place, so the rate b of vaccine protection currently used is b_v/p.

We first explore whether there is consistency in eligibility criteria for phase Ⅲ vaccine trials in non-public health and public health areas.

According to the equivalence relation between the formula b_v/p ≥50% and the comprehensive protection rate b_v+p ≥ b_p+(1-b_p) 50%, we can know that:

(1) In areas b_p= 0, where no public health measures are taken, qualified vaccines as long as met b_v+p ≥ 50%. That is, as long as vaccines and public health measures provide protection to 50% of the susceptible population.

(2) In areas where public health measures are taken b_p> 0, qualified vaccines must be met b_v+p ≥ b_p+(1-b_p) 50% = 50% (1+b_p). Moreover, the qualified threshold 50% (1+b_p) of the comprehensive protection rate b_v+p in the area taking public protection measures increases with the increase of the protection rate b_p of public health measures.

From the above analysis, it can be seen that setting requirements b_v/p ≥ 50% as vaccine qualifications will lead to higher vaccine qualification requirements for phase Ⅲ trials in areas where public health measures are taken than in areas where public health measures are not taken. And the higher the protection rate of public health measures, the more difficult it is to do phase Ⅲ trials to meet the requirements of qualification [7-10]. Therefore, it is inappropriate to adopt b_v/p ≥ 50% as the eligibility criteria for vaccine qualification.

Feasibility of comprehensive protection rate b_v+p as evaluation Indicator of vaccine protection effect in phase III trial

Since the public health measure protection rate b_p is involved in Equation (9), the calculation methods and steps are given below.

(1) Calculate the incidence of diseases in each country based on the number of cases and population data in each country;

(2) According to the conclusion of Pfizer's phase Ⅲ trial of COVID-19 vaccine involving 43,000 volunteers:

Efficacy was consistent across age, gender, race and ethnicity demographics [3], we may suppose that r_n in the different regions are equal. To ensure that b_p ≥ 0 holds, we rank the incidence data from big to small, and approximately taken r_n as the maximum incidence rate of each country. So the incidence of other countries is r_p≤ r_n , Thus, according to the formula (2) b_p ≥ 0.

(3) Use formula (2) to calculate the protection rate b_p of public health measures in the world.

It is important to note that the use of the data with the highest incidence as incidence rates in countries where public health measures have not been taken is only for research purposes, and does not mean that the country with the highest incidence has not taken public health measures.

From the calculation method of b_p, as the countries have adopted the unified r_n, b_pis therefore comparable, its size can reflect the protective effect of national public health measures. Since the comprehensive protection rate also takes the same r_n as the incidence rate under no public health measures, therefore, the b_v+p of different countries are comparable.

Since b_p is the data computed statistically based on the number of patients in a country or region, its accuracy is generally high, so the error is therefore negligible and according to the formula (9), it is obtained that

So there is

It follows that,

When b_p→0

That is, when the intensity of public health measures is weak, the relative error of the comprehensive protection rate b_v+p is almost equal to that of the current protection rate b_v/p.

It can be derived from formula (17), that

It can be seen that when b_p→1, i.e. 1−b_p →0, is much smaller than that of b_v/p. That is, in areas where the protection rate of public health measures is high, the comprehensive protection rate b_v+p shows better computational stability than the current protection rate b_v+p.

Due to the comparability and computational stability of b_v+p, it is feasible to use b_v+p as an evaluation Indicator for the protective effect of different vaccines in phase Ⅲ trials.

For verifying the impact of public health measures on the protection rate of phase Ⅲ clinical trials of the vaccine now adopted, the data from phase Ⅲ clinical trials of COVID-19 vaccine are used here.

We calculated the public health measures protection rates and the comprehensive protection rate for the region where the phase Ⅲ trials were conducted, using WHO website data on cumulative COVID-19 patients worldwide (as of Saturday, 11 January 2021) and population data [4] (2020 data on WHO website).

Because the Czech Republic was the country with the highest incidence rate at that time, its incidence rate reached 7.76%, so r_c/n≈ r_n7.76 % is selected (Table 2).

Table 2: Data from phase Ⅲ trials of various COVID-19 vaccine companies.

Company	Test area	Cumulative COVID-19 patients	Total population (104)	(%)	(%)	(%)
Moderna5	US	22917334	33100.3	10.8	94.5	95.1
AstraZeneca6	US UK BR	34095473	61144.8	28.1	95	96.4
Pfizer3	US AR BR DE	34675092	67254.2	33.5	95	96.7
Novavax7	UK	3072349	6788.6	41.6	89.3	93.8
Gamaleya8	RU	3401954	14593.4	70.0	91.4	97.4
Sinovac8	BR	8105790	21255.9	50.9	50.65	75.8
	CL	641923	1911.6	56.7	67	85.7
	TR	2326256	8433.9	64.4	83.5	96.9
	ID	828026	27352.4	96.1	63.5	98.6
Beijing Biology8	AE BH EG JO	783137	12412.9	91.9	78.1	98.2
Wuhan Biology8	AE BH EG JO	783137	12412.9	91.9	72.8	97.8
CanSino BIO9	AR CL RU MX PK	7792546	56007.1	82.1	63.7	93.5
Johnson & Johnson10	US	22917334	33100.3	10.8	72	75.0
	AR BR CO CL PE MX	14816050	48965.9	61.0	66	86.7
	ZA	1231597	5930.9	73.2	57	88.5

The data in Table 1 can verify the 2 conclusions in the text.

The better the public health measures, the more likely it is to cause a significant decline in the value of the indicators b_v/p currently used

As can be seen from Table 1, the highest protection rate b_v/p in phase Ⅲ trials was only 78.1% in areas with high protection rates from public health measures (b_p> 80%). Vaccines with a protection rate b_v/p of 90%, but the maximum public health protection rate b_p in their phase Ⅲ trial areas is only 70%. The modena, Astrazeneca, and Pfizer vaccines were tested in areas where public health protection rates were ranging from 10.8% to 33.5%, and they all got high b_v/p of more than 90%. The vaccines of Sinovac, Beijing Biobiology and Wuhan Biobiology were tested in areas with 90% public health protection, but the highest protection rate b_v/p was 78.1%.

CanSino BIO and Johnson and Johnson produce a single needle vaccine. The phase Ⅲ trial of the CanSino BIO vaccine was conducted in an area with a high public health protection rate of 82.1%, and also had a lower b_v/p of 65.7%.

Datum of Johnson and Johnson are even more convincing. Because Johnson and Johnson tested the same vaccine in all three regions. The public health protection rate b_p of the three regions were 10.8%, 61.0%, and 73.2%, respectively. The results of b_v/p were 72%, 66%, and 57%. It is clear that the higher the rate of protection from public health measures in the region where the phase Ⅲ vaccine trial was conducted, the lower the data of b_v/p.

To sum up, b_v/p is not suitable to be used as an evaluation indicator of the advantages and disadvantages of different vaccines because it is greatly affected by the public health protection rate.

It is feasible to use b_v+p as an evaluation indicator of the protective efficacy of vaccine phase Ⅲ trials

The fact that b_v/p trialed in the three regions with the same vaccine of Johnson and Johnson vaccine in the three regions of 72%, 66%, 57%, respectively, resulted arge difference in values, the maximum ratio minimum difference reached 26%. Note that the impact of the public health measures, the comprehensive protection rates b_v+p were 75%, 86.7%, and 88.5%, respectively. The difference of these values decreased. The difference of the maximum and minimum was 18%. This shows that the comprehensive protection rate b_v+p is more comparable.

Vaccine produced by Modena, AstraZeneca, Pfizer, Novavax, Gamaleya, Sinovac, Beijing Biology and Wuhan Biology all have 90% or more protective effects. The single needle vaccine produced by CanSino BIO and Johnson and Johnson also had a comprehensive protective effect of 93.5% and nearly 90%, respectively.

Whether or not to vaccinate a particular vaccine locally is related to the significance of the incremental protection provided by the vaccine beyond the protection provided by local public health measures. If it is significant, vaccination may be considered; otherwise, not.

Under the premise of b_v+p being identified as the evaluation Indicator of the protective effect of different vaccines in phase Ⅲ trials, a judgment method with significant protective effect is presented here.

Assuming that the comprehensive protection rate calculated in the region where the phase Ⅲ vaccine trial is b_v+p0 , and the protection rate of public health measures in the region where the vaccine is to be inoculated is b_p1 , we use

As the significance test index of vaccine protection. When I ≥ε , the vaccine was considered to have significant protective value, and ε (0 ≤ε ≤1) is the threshold. How to determine an appropriate threshold involves local political, economic and other issues, which will not be studied here.

Based on the above analysis of evaluation indicators for phase Ⅲ vaccine trials and discussion of vaccine selection criteria, the process of thevaccine selection in each region is suggested as follows:

Step 1: Calculate the significance test index I for the vaccine in the area. When I ≥ε ,the vaccine was considered to have a significant local protective value.

Step 2: Select the most suitable local vaccine product for vaccination based on b_v+p0 of the vaccines that have passed the significance test index.

Based on the above theoretical analysis and relevant data verification on the phase Ⅲ trial of COVID-19 vaccine, we obtained the following conclusions:

The current vaccine protection rates and eligibility criteria for tested vaccines are heavily influenced by public health measures in the region where the phase Ⅲ vaccine is being tested, and therefore are not suitable as indicators for evaluating the effectiveness of protection between vaccines and eligibility criteria for testing vaccines.

1. It is suggested to use the comprehensive protection rate as the evaluation indicator of the effectiveness of inter-vaccine protection.

2. It is recommended to first test the significance of vaccine protection effect, and then compare the overall protection rate to determine which company to vaccinate in each region.

1. Halloran ME, Longini IM (2010) CJ Design and Analysis of Vaccine Studies.
2. Ruyun Wang (2021) Study on uncovering blindness requirements and protection rate in the midterm of phase iii clinical trial of Covid-19 vaccine. Adv App Math 10(3): 781-789.

   [Crossref] [Google Scholar]

3. Pfizer and BioNTech conclude phase 3 study of covid-19 vaccine candidate, meeting all primary efficacy endpoints.
4. WHO coronavirus (COVID-19) dashboard.
5. Moderna's COVID-19 vaccine interim readout suggests 94.5% efficacy.
6. Summary of product characteristics.
7. Novavax COVID-19 vaccine demonstrates 89.3% efficacy in uk phase 3 trial.
8. Jingesi M, Biyun Z, Ping Y (2021) Evaluation of SARS-CoV-2 vaccine based on phase Ⅲ clinical trials and real-world studies. Herald Med 40(9): 1159-1168.

   [Crossref] [Google Scholar]

9. Convidecia phase III results published in the lancet.