Revocation Statuses on the Internet
RRevocation Statuses on the Internet
Nikita Korzhitskii and Niklas Carlsson
Link¨oping University, Sweden
Abstract.
The modern Internet is highly dependent on the trust com-municated via X.509 certificates. However, in some cases certificates be-come untrusted and it is necessary to revoke them. In practice, the prob-lem of secure certificate revocation has not yet been solved, and today norevocation procedure (similar to Certificate Transparency w.r.t. certifi-cate issuance) has been adopted to provide transparent and immutablehistory of all revocations. Instead, the status of most certificates can onlybe checked with Online Certificate Status Protocol (OCSP) and/or Cer-tificate Revocation Lists (CRLs). In this paper, we present the first lon-gitudinal characterization of the revocation statuses delivered by CRLsand OCSP servers from the time of certificate expiration to status dis-appearance. The analysis captures the status history of over 1 millionrevoked certificates, including 773K certificates mass-revoked by Let’sEncrypt. Our characterization provides a new perspective on the Inter-net’s revocation rates, quantifies how short-lived the revocation statusesare, highlights differences in revocation practices within and between dif-ferent CAs, and captures biases and oddities in the handling of revokedcertificates. Combined, the findings motivate the development and adop-tion of a revocation transparency standard.
The modern Internet uses the Web Public-Key Infrastructure (WebPKI) as afoundation to establish trust between clients and servers. In WebPKI, Certifi-cate Authorities (CAs) issue signed X.509 certificates that verify the mappingbetween public keys and public distinguished names, such as domain names.In certain cases (e.g., a private key compromise, owner’s request, or misis-suance by a CA), certificates must be revoked; i.e., rendered invalid. To protectclients and servers from the use of revoked certificates, WebPKI supports severalrevocation protocols. Currently, revocation statuses of most certificates can beobtained via Online Certificate Status Protocol (OCSP) servers [28], but someCAs continue to support the traditional Certificate Revocation Lists (CRLs) [6]as a complementary option. However, these pull-based protocols raise many se-curity, privacy, and performance issues. Therefore, many browser vendors do notutilize the protocols [23], but instead, they push a proprietary set of revocationsto the users [2,11]. Yet, these push-based revocation mechanisms have their ownlimitations, which leave secure certificate revocation an open problem [4].Furthermore, as of today, there does not exist any standardized mechanismin place (similar to Certificate Transparency (CT) [20,14,30] w.r.t. certificate
Author’s edition. Accepted to Passive and Active Measurement Conference 2021. Cite as
In Proc. PAM 2021 . a r X i v : . [ c s . N I] F e b ssuance) to provide an immutable history of all revocations and correspondingrevocation reasons. Consequently, there is no ability to easily study and detectrevocation-related misbehavior by CAs (e.g., advertisement of wrong, or contra-dictory revocation statuses). While many novel WebPKI extensions, revocationprotocols, architectures, and transparency schemes have been proposed to ad-dress this issue, none have been adopted so far [4]. Instead, we observe that theinformation about revocations is sparse and most revocation statuses disappearsoon after certificate expiration.In this paper, we make a case for revocation transparency by presenting anovel characterization study of the revocation rates on the Internet, the post-expiry life of revocation statuses, and the status-handling practices across CAs.First, we present a measurement methodology that allows us (i) to obtain nearlyall revocations performed for the set of certificates expiring during a time win-dow, and (ii) to track the certificate status (using both OCSP and CRL) of suchsample sets over 100-day periods, starting at their respective expiration dates .Second, we track all certificates from the Censys dataset [10] that expiredbetween Mar. 2, 2020, and Apr. 1, 2020, and that were valid with respect toApple’s, Microsoft’s, or Mozilla’s root stores. This time period (see Figure 1)is particularly interesting since the measurement was done prior to and duringthe mass-revocation event in which Let’s Encrypt (LE), the largest CA, initiallyannounced to revoke over 3 million certificates [22] due to a CAA-recheckingbug, but in the end, they revoked only 1.7 million certificates [21].Third, and most importantly, we characterize the revocation-status-handlingpractices across CAs, including status lifetimes beyond the expiration date andhandling differences across CAs and certificate types. We identify classes of be-haviors, compare and contrast practices of different CAs, find revocation biasesamong different sets of certificates, and look closer at some odd CA behaviors(e.g., certificates that switch back to a “Good” status after being advertised as“Revoked”). Across our analysis, we observed highly heterogeneous behaviorsamong CAs and quick disappearance of revocation statuses. This highlights thelack of a global revocation transparency standard that would otherwise help toidentify and improve odd revocation behaviors, similarly to CT, with its effecton the issuance process. Finally, we share our dataset [19]. Outline:
After a brief overview of revocation protocols (Section 2), wepresent our methodology (Section 3) and characterization results (Section 4).Finally, related work (Section 5) and conclusions (Section 6) are presented.
The two primary revocation protocols that CAs typically use are the following. – Online Certificate Status Protocol (OCSP):
Using OCSP, a client canrequest the status of a certificate by providing a serial number and thehashes of the issuer’s name and key. The CA-Browser forum requires signed Currently, CAs must maintain revocation statuses only until certificate expiration [1]. ig. 1.
Timeline of the measurement. responses to be valid for at least 8 hours, and at most 10 days [1]. OCSPcan be used in different ways. For example,
OCSP stapling allows statuses tobe delivered by a web-server, and the
OCSP Must-staple extension preventsa client from making OCSP requests on their own and enforces a hard-failpolicy if the status was not delivered by the web-server. The Must-stapleextension is not widely adopted yet [5]. Instead, most browsers typicallyaccept a certificate if they are unable to obtain revocation information [23]. – Certificate Revocation List (CRL):
CAs maintain signed lists with theserial numbers of revoked certificates, and optionally, corresponding invali-dation dates and reason codes for the revocations. CRLs can also be aug-mented using several extensions (e.g., CRL number, Authority Key Identi-fier, etc.) [6]. CRLs are required to be reissued at least once every 7 days [1].Due to the security, privacy, and performance issues with OCSP and CRL,many browser vendors have disabled the above pull-based revocation protocols;instead, they periodically push limited sets of revocations to the clients (e.g.,via software updates) [2,11]. However, this approach has some limitations; e.g.,a delay introduced by scheduled updates, and a small coverage of all existingrevocations.WebPKI lacks revocation transparency, and no mechanism similar to CT hasbeen adopted yet. In fact, CAs are not required to maintain revocation statusesfor certificates beyond their expiration date [1], and as we show in this paper,most of the time, revocation statuses stop being advertised shortly after certifi-cate expiration. The lack of a transparent and immutable history of revocationscomplicates keeping CAs accountable for their revocation mishandling.
We conducted a four-phase measurement campaign (see Figure 1).
1. Preparation:
In the first phase, we collect all X.509 certificates (withtheir parent certificates) found in CT logs [20] and active scans that expirewithin a period starting from Mar. 2, 2020, to Apr. 1, 2020, using Censys [10].For the analysis, we only select certificates that are valid with respect to Apple’s,Microsoft’s, or Mozilla’s root stores [18]. From these certificates, we extract allOCSP responder URLs (used in phases 3+4) and CRL URLs (used in phase 2). able 1.
Summary of the studied certificates. (LE – Let’s Encrypt).
Certificates Event LE Rest LE Other CAs All
Non-revoked – 36,755,317 11,496,607 48,251,924Revoked 773,128 129,552 174,712 1,077,390Revocation rate 100% 0.35% 1.50% 2.18%
For every remaining certificate, we then schedule an OCSP first pass (phase 3) 22hours before its expiration , and for every observed CRL, we schedule periodicCRL requests (phase 2).
2. CRL follow-up:
During the second phase, we regularly (every 12 hours)fetch all CRL lists using the URLs extracted in the first phase.
3. OCSP first pass:
In the third phase, we perform an OCSP status lookupfor each certificate 22 hours before it expires. If a certificate is found to be revokedduring its first pass, it gets scheduled for follow-up checks every 12 hours (phase4). In the case of an OCSP timeout or an error, the first pass is retried everyminute until a revocation status is obtained or the certificate is expired.
4. OCSP follow-up:
In the fourth phase, the revocation status of ev-ery revoked expired certificate is fetched every 12 hours for 100 days (sincethe first pass of each individual certificate). We separate OCSP responses intofour types: “Good”, “Revoked”, “Unauthorized”, and “Unknown”. The firsttwo types (“Good” and “Revoked”) are cryptographically-signed responses thatdefinitively specify the status of a certificate. The third type (“Unauthorized”) isan unsigned plaintext response. The final category (“Unknown”) contains signed“Unknown” statuses (that some CAs deliver) and other unsigned responses.
External effects on the sampling rate:
Between May 12, 2020, and May19, 2020, parallel processes running at our server have temporarily increasedthe average OCSP inter-request time from 12 hours up to 21.7 hours. Exceptfor this short period, the average OCSP inter-request time was consistently 12hours ± a few minutes, up until June 21, 2020. Between June 21, 2020, and theend of our measurement period on July 20, 2020, the average inter-request timewas roughly 24 hours. Neither of the periods with increased OCSP inter-requesttimes took place during the first month after the expiration date of any of thecertificates; hence, the effects do not impact our conclusions. In total, we collected OCSP status information for 49 million certificates. Ta-ble 1 provides a breakdown based on whether a certificate was revoked or not,whether the certificate was issued by Let’s Encrypt (76.3% of the certificates) ora different CA (23.7%), and whether a Let’s Encrypt certificate was part of theabove-mentioned mass-revocation event (1.57%). For us to consider a certificatemass-revoked it needed to be (i) on the list of 3M certificates that Let’s Encrypt The interval of 22 hours (slightly less than 24 hours) was selected for performancereasons, after the initial evaluation of our measurement framework.
Mar. 2 Mar. 9 Mar. 16 Mar. 23 Mar. 30 C e r t i f i c a t e s / da y Other CAsLet's Encrypt (non-mass-revoked)Let's Encrypt (mass-revoked)
Fig. 2.
Revoked certificates with a given expiration date. publicized for the event [22] and (ii) to be revoked at the time it expired. Wealso found that 297,242 certificates from the list, with expiration dates fallingon our first pass period, have never been revoked.The timing of the mass-revocation event is particularly interesting since itprovides a concrete example of the impact that such events can have on therevocation rate and the lifetime of revocation statuses. Finally, we note thatthe certificates affected by recent mass-revocation events have been disclosedthrough website postings of arbitrarily formatted datasets [22,8,9].While the non-mass-revocation-rate of Let’s Encrypt was much smaller thanfor the other CAs (0.35% vs 1.50%), the mass-revocation event increased Let’sEncrypt’s revocation rate for this period up to 2.40%. The effect is perhaps mostnoticeable when looking at the number of revoked certificates per day, based ontheir day of expiry, as shown in Figure 2. Here, starting from Mar. 5, 2020, wecan see the impact of the certificates associated with the mass-revocation event(gray in the figure). The other two classes of revocations (blue, orange) remainedrelatively stable throughout the measurement period.We found large variations in the revocation rates of different CAs. Figure 3shows the number of revoked (blue) and non-revoked (gray) certificates, brokendown per CA. The orange markers show the number of revoked certificates listedin the CRLs, in addition to OCSP servers (discussed in Section 4.4). Here, weshow all CAs with at least 100 revoked certificates in our dataset, ranked fromthe one with the most revocations to the one with the least. We also include the“other” category that combines the results for all other CAs. While most CAshave much fewer revoked certificates than non-revoked certificates, there are no-table exceptions. Five CAs even had more revoked than non-revoked certificates:Actalis (92.5%), nazwa.pl (66.4%), SwissSign (59.9%), Plex (73.7%), Digiden-tify (100%). Among the most popular CAs (i.e., CAs with the highest gray/bluebars), GoDaddy also stands out with 34.5% being revoked before expiry.
The revocation statuses provided by OCSP servers often change from “Revoked”to some other status soon after certificate expiry. Figure 4 shows the time thatthe status remained “Revoked” after the revoked certificates had expired. Here,we filter out any temporary OCSP responses (e.g., unauthorized, unknown) andtimeouts whenever we obtained at least one more “Revoked” response.
Quickly disappearing revocation statuses:
Figure 4(a) shows the empir-ical Cumulative Distribution Functions (CDFs) for four classes of revoked certifi- Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s C e r t i fi c a t e s RevokedNon-revokedRev/CRL
Fig. 3.
Per-CA breakdown of the number of revoked (blue) and non-revoked (gray)certificates in the dataset. Revoked certificates found in CRLs are shown with × . CAsare ordered by the number of revoked certificates, in descending order from left toright. In the following figures, the order is preserved. CD F Time remained 'revoked' after expiry (days)
Other CAsOther CAs (only EV)Let's Encrypt (non-mass r.)Let’s Encrypt (mass-revoked) (a) CDFs Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r f i e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s P e r c en t age ( % ) > 7 days > 30 days (b) Per-CA breakdown Fig. 4.
Time that the revoked certificates remained revoked after the expiration. cates: 2 for Let’s Encrypt certificates (mass-revoked and non-mass-revoked) and2 for certificates by other CAs (with and without Extended Validation (EV)). Allcertificates by Let’s Encrypt changed status within 3 days of expiration. Theirmass-revoked certificates (gray) had longer status change times than the non-mass-revoked certificates (orange). The CDFs for the other CAs are relativelyflat from about two weeks to 100 days. (Note the logarithmic y-axis.) On an en-couraging note, the certificate class with the most long-lived revocation statusesis Extended Validation (EV) certificates (black). This class of certificates shouldtypically endure the most scrutiny.
Some CAs keep the state longer:
Figure 4(b) shows the fraction of thecertificates issued by different CAs that maintained the revoked status for atleast 1 week or 30 days. While many CAs maintained “Revoked” state for veryshort time periods after certificate expiry (e.g., blue CDF in Figure 4(a) and CAswithout any bars in Figure 4(b)), most of the CAs that did keep the “Revoked”state beyond a week also kept this state beyond 30 days (brown bar).
Status response overview:
For the revoked certificates, we performed morethan 207 million OCSP status requests. Table 2 provides a per-category break-down of the individual responses (“Resp.” in the table) and the fraction of cer-tificates (“Certs”) with at least one such response.All certificates started as “Revoked” and most eventually changed to an unau-thorized response (100% of Let’s Encrypt certificates and 76.43% of other CAs’certificates). While we only had timeouts for 0.04% of the status requests, thedifferences between the number of affected certificates were substantial betweenCAs: only 0.07% of the Let’s Encrypt certificates had at least one timeout, com- able 2.
Summary of different types of OCSP status responses.
Revoked Unauthorized Unknown Timeout Good
Certs Resp. Certs Resp. Certs Resp. Certs Resp. Certs Resp.Mass rev. (LE) 100.00 2.83 100.00 97.10 12.37 0.07 1.43 0.01 - -Non-mass. LE 100.00 2.17 100.00 97.76 11.95 0.07 1.54 0.01 - -Other CAs 100.00 13.19 76.43 74.06 13.51 12.35 13.98 0.22 0.34 0.19Total 100.00 4.43 96.18 93.43 12.50 2.07 3.48 0.04 0.05 0.03 Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r f i e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s P e r c en t age ( % ) Unauthorized Unknown Revoked (only) Other
Fig. 5.
Dominating status change behavior of different CAs. pared to 13.98% of the other CAs’ certificates. These fractions are non-negligible,since most browsers soft-fail on an OCSP timeout and continue to establish apotentially-insecure connection. A concerning observation is that 589 certificatesissued by 13 CAs (0.34% in the other CA category) switched from “Revoked”status to “Good” (65,791 responses in total).
Most frequent behaviors:
Usually, public certification practice statementsof CAs guarantee revocation status preservation for non-expired certificates, butdo not specify the CAs’ actions after that [15,33,13]. We next look at the most fre-quent CA behaviors. For this analysis, we filtered out temporary status changeswhenever we observed the original state again. With this filtering, we observedthe following dominating behaviors. – Let’s Encrypt almost always transition statuses from “Revoked” to “Unau-thorized”. This behavior was observed for 772,042 (99.86%) of the mass-revoked certificates and 129,400 (99.88%) of the other certificates revoked byLet’s Encrypt. A possible explanation for this behavior is that they respondwith code “Unauthorized” as soon as the status record has been removed [7].Let’s Encrypt’s current certification practice statement only guarantees that“OCSP responses will be made available for all unexpired certificates” [15]. – Among the other CAs, we observed three dominating behaviors: 133,276(76.28%) cases where the CA simply transitioned to “Unauthorized” (likeLet’s Encrypt), 21,816 (12.49%) cases where the status always changed to“Unknown”, and 18,660 (10.68%) cases where the “Revoked” status re-mained for the duration of our measurement period.Figure 5 breaks down the use of the dominating status change behaviors em-ployed by the different CAs. In addition to the three behaviors mentioned above,we include the “other” behavior category. Most CAs have a dominating behaviorthat they employ for almost all of their certificates: 15 (out of 26) CAs almostalways switch from “Revoked” to “Unauthorized” (pink bars), 9 (out of 26)CAs almost always keep the “Revoked” status for the full 100 day period, Ac-talis mainly switch certificates from “Revoked” status to “Unknown” (except for1 cases, when the statuses were switched to “Good”, following the intermedi-ate “Unknown” status), Digidentify (who revoke all certificates) always start totimeout, and Japan Registry always switches statuses to “Good”. As expected,the “other CA” category (not explicitly listed), contains a mix of behaviors.These results demonstrate the lack of a standard practice w.r.t. revocation sta-tuses after certificate expiration. We have also observed some small differencesin the weekly status-change patterns between CAs; however, compared to thedifferences in issuance timing, these differences are very small. See Appendix A.
Special cases with the “Good” status:
589 revoked certificates switchedto status “Good”. In almost all cases the servers kept the “Good” status untilthe end of the measurement period. In 349 of these cases, the status changeddirectly from “Revoked” to “Good” and in 91 cases an intermediate “Unknown”status was observed. All these cases provide strong motivation for transparentlong-term recording of revocation information.We note that Let’s Encrypt and most of the other big CAs did not haveany cases with the above strange behavior. Of the CAs with at least 100 revoca-tions, only the following CAs had such cases: GoDaddy (117 cases), Actalis (91),Starfield (9), Entrust (5), and Japan Registry (135). Other CAs (not listed inour figures) with many cases include: “National Institute of Informatics” (91),“SECOM Trust Systems” (70), “ACCV” (54). (The rest of the non-listed CAshad five or fewer revoked certificates changing to status “Good”.) Finally, a fewcertificates in this category stood out more than the others. For example, thelist included three EV certificates: one by Entrust for “JPMorgan Chase andCo” (“Revoked” → “Good” → “Revoked”), one by GoDaddy for “DelmarvaBroadcasting Company” (“Revoked” → “Unauthorized” → “Good”), and oneby Actalis for “Pratiche.it” (“Revoked” → “Unknown” → “Good”). Otherwise,all the certificates in this class include RSA keys with the following key lengths:1024 (9), 2048 (579), and 4096 (1). Furthermore, only 123 (out of 589) hadSigned Certificate Timestamps (SCTs) embedded. We contacted all CAs withthe above behavior. A summary of the responses is provided in Appendix B. We have found that the revoked certificates typically havelonger validity periods. Figure 6(a) shows CDFs of the validity periods for bothrevoked (blue) and non-revoked (gray) certificates for all CAs other than Let’sEncrypt. (Since Let’s Encrypt always use a 90-day validity period, we kept thesecertificates separately.) Here, we note a clear shift between the two curves.Figures 6(b) and 6(c) provide a similar comparison of the (b) revoked and(c) non-revoked certificates on a per-CA basis. Here, we plot the fraction of cer-tificates with validity periods longer than 89 days, 90 days, 1 year (365 days),and 2 years (720 days), respectively. These choices are based on the observationthat many CAs use validity periods of either 90 days or 398 days (e.g., steps inthe CDFs in Figure 6(a)). For almost all CAs, the fraction of certificates withlong validity periods (darker colored bars) is larger among the revoked certifi-cates (Figure 6(b)) than among the corresponding CA’s non-revoked certificates CD F Validity period
Let's EncryptOther CAs; revokedOther CAs; non-revoked (a) CDFs of certificate validity Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r f i e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s P e r c en t age ( % ) >89 d >90 d >1 yrs >2 yrs (b) Revoked certificates Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s P e r c en t age ( % ) >89 d >90 d >1 yrs >2 yrs (c) Non-revoked certificates Fig. 6.
Validity periods for different categories of revoked and non-revoked certificates. (Figure 6(c)). This is in part an effect of CA/Browser Forum conventions [1] anddecisions by individual browsers [3,12,24] forcing CAs to use shorter certificatevalidity periods. Another reason is that older certificates have had more time tobecome compromised. It could also be an indication that CAs apply increasinglystricter security policies (e.g., to comply with CT [20]).
Public key types:
The modern WebPKI relies on EC (Elliptic Curve) [17]and RSA (Rivest–Shamir–Adleman) [26] public-key cryptography. Here, we com-pare the use of different key types and key lengths. While RSA 2048 is thedominating public key among both revoked (90.44%) and non-revoked (80.81%)certificates, there are significant differences in the revocation rates of certificatesincluding different key types. For example, certificates with RSA 3072 (4.55% re-vocation rate), EC 521 (80.49%) and RSA with key lengths other than the threemost common lengths (6.67%) all have revocation rates well above average. Incontrast, EC 256 (0.14%), EC 384 (0.62%) and RSA 4096 (1.48%) all have re-vocation rates below average. These differences are also present when looking atcertificates of Let’s Encrypt and other CAs separately. Table 3 summarizes theoverall revocation rates (column 4) for each key type (column 1) and the keyusage distributions seen for each of the three certificate groups: Let’s Encrypt(columns 5 vs 6 vs 7), other CAs (columns 8 vs 9), and the aggregate over allcertificates (columns 2 vs 3). Above/below average revocation rates are shownwith red/green color (column 4) and bold text indicates the sub-group withthe highest relative representation (on a per-group basis). With this annotation,higher revocation numbers (bold) mean revocation rate above average (red).
SCT and EV usage:
To measure the CT compliance we looked at the useof Signed Certificate Timestamps (SCTs). While all certificates issued by Let’sEncrypt have embedded SCTs, other CAs do not always embed the timestamps.Furthermore, among the certificates issued by other CAs, the fraction of certifi-cates that do not contain SCTs was much greater among the revoked (10.04%) able 3.
Key usage comparisons based on revocation vs non-revocation sets.
All certificates Let’s Encrypt (%) Others (%)Key type Revoked Non-revoked Revoked M-rev. Rev. Non. Rev. Non.RSA 2048 974,405 ( ) 38,996,600 (80.82%) 2.44% 88.75 ) 285,636 (0.59%) 4.55% ) 1.48% 8.70 8.85 ) 406 (0.00%) 6.67% - - ) 0.14% 0.36 0.45
EC 384 4,436 (0.41%) 712,770 ( ) 0.62% 0.52 0.34 ) 16 (0.00%) 80.49% - - - CD F Normalized lifetime
RevokedCRL entry last seen (a) Revocation timing -6 -5 -4 -3 -2 -1 CD F CCD F CRL Entries
CDF (all)CCDF (all)CDF (max t )CCDF (max t ) (b) Entries per CRL measurement Fig. 7.
Distributions for measured CRLs. than non-revoked certificates (1.91%). In addition to having longer validity pe-riods, some of the older non-expired certificates lack embedded SCTs. Ownersand issuers of these certificates may be replacing them with certificates that bet-ter meet recent browser requirements [3,25]. We have also observed significantlyhigher revocation rates among EV certificates. For example, 1,890 (10.77%) outof the 17,544 observed EV certificates were revoked. Furthermore, for CAs otherthan Let’s Encrypt, 1.08% of the revoked certificates are EV certificates and0.14% of the non-revoked certificates are EV certificates.
For the 2,190 CRL URLs extracted from the certificates of interest, we collected643,860 CRL snapshots. Combined, these snapshots included CRL entries for169,911 (15.8%) of the revoked certificates found using OCSP. Let’s Encrypt’sdecision not to implement CRL contributes to the small fraction. Here, we focuson the certificates with at least one CRL entry and one OCSP “Revoked” status.
Timing analysis:
On average, revocation statuses disappear even fasterfrom CRL lists than from OCSP responders. For example, only in 26.5% of thecases did we observe the revocation status in the CRLs after the expiration dateof the certificates, and only for 2.9% did we observe the status being preservedlonger than a week after expiration. This may be an attempt to reduce the sizeof the CRLs. However, since the majority of the revocations happen early in thelifetime of the certificates (e.g., the median normalized lifetime is 13.8%) thereis still a significant time period over which certificates are included in the CRLs.This is illustrated in Figure 7(a), which shows the normalized timing of revoca-tions and when the CRL entries are last observed in our dataset. Here, all valuesare normalized relative to the total intended validity period (i.e., “NotBefore”and “NotAfter” corresponds to the values 0 and 1, respectively). As implied byLittle’s law, the average size of a CRL (e.g., measured as entries per CRL) is Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s P e r c en t age ( % ) No CRL (--)No reason (19.6%)Cess. operation (65.4%) Superseded (13.4%)Affiliation change (0.73%)Hold (0.02%) Priv. withdrawn (0.57%)Unspecified (0.03%)Key compromised (0.22%)
Fig. 8.
Per-CA breakdown of CRL-listed reasons for revocation. equal to the average time that the entries remain in the CRL (e.g., measured indays) times the average rate that certificates are being added to the CRL (e.g.,revocations per day), CRL sizes therefore easily become very large. Indeed, theaverage CRL size was 7,362 entries and the largest CRL contained 1,139,538entries at its peak. Figure 7(b) shows CDFs and CCDFs for both individualmeasurements (all) and when using the observed peak size (max t ). We also ob-served some CRLs that did not appear to delete entries and roughly 0.94% ofthe certificates remained in the CRLs for the full duration of our measurement. Revocation reasons:
Figure 8 breaks down the percentage of certificates forwhich (i) we did not find any CRL entry (blue), (ii) we found CRL entries with-out revocation reason (gray), or (iii) we found a revocation reason for (orange,brown, black) on a per-CA basis. For simplicity, we only show the dominatingreasons using colors (orange, brown) but provide the overall percentages (overall certificates with CRL entries) in the figure key. The four dominating CRLbehaviors that we observed were: (i) some CAs did not use CRLs (Let’s En-crypt, Plex) or only used it to a limited degree (e.g., Sectigo, FNMT-RCM), (ii)17 CAs used CRLs for the majority of their revocations but did not provide anyrevocation reason, (iii) three CAs almost always used “Cessation Of Operation”as revocation reason (GoDaddy, Google, Starfield), and (iv) three CAs almostalways specified “Superseded” as the revocation reason.Overall, most revoked certificates are not included in CRLs and 19.6% of CRLentries contain no revocation reason. Our results show that the practices of CAsare highly heterogeneous and revocation statuses are not persistent; thus, weargue that the Internet would benefit from a revocation transparency standard.
A number of studies have measured the revocation rates on the Internet. Liuet al. [23] performed several IPv4 HTTPS scans and found that a large fractionof served certificates was revoked (8%), while CRLSets [11] by Google was onlycovering 0.35% of all revocations. Chung et al. [5] evaluated the performanceof OCSP responders by sending OCSP requests from geographically separatedlocations. They concluded that OCSP responders were not sufficiently reliableto support
OCSP Must-staple extension. Zhu et al. [34] found OCSP latencyto be “quite good”, and showed that 94% of OCSP responses are served usingDNs. Moreover, only 0.3% of certificates were found to be revoked at that time(2015). Smith et al. [32] propose an efficient scheme to disseminate revocations.In the process, they measured revocation rates and found that in the absenceof a mass-revocation event, the revocation rate on the Internet was 1.29%. Thisis similar to what we observed. The above works perform OCSP status checksbefore certificate expiration, while we check the certificates the day before theirexpiration and onward. Revocation effectiveness at the code-signing PKI wasmeasured in [16], and a number of security problems related to revocations wereidentified. A recent survey and a comprehensive framework for comparison ofimplemented and proposed revocation/delegation schemes are provided in [4].
Other community efforts and data sources:
The CA/Browser forumspecifies some requirements that motivated our measurement design, includingthe requirement that “revocation entries on a CRL or OCSP Response MUSTNOT be removed until after the Expiry Date of the revoked Certificate” [1]. Weused the Censys search engine, backed by Internet-wide scanning [10], to obtainall certificates for our study. Some other online services also provide revocationstatuses. For example, crt.sh [31] selectively obtains CRL statuses for certificates;but, updates are not regular. Until late Aug. 2020, Internet Storm Center [27] wasregularly fetching several CRLs; however, they did not monitor all CRLs presentin our dataset and did not capture the mass-revocation by Let’s Encrypt.
In this paper, we have presented the first characterization of the revocation statusresponses provided by OCSP and CRL responders from the time of certificate ex-piration and beyond. We described a measurement methodology, which allowedus to look at the revocation rates on the Internet from a new perspective; wequantified how short-lived the revocation statuses are, and highlighted differencesin status handling practices of different CAs. We found that most CAs remove re-vocation statuses very soon after certificate expiration. Some CAs do not provideCRL entries for all revoked certificates and/or remove entries from the CRLs be-fore certificate expiration. The CA-dependent differences highlighted throughoutthe paper (e.g., revocation status lifetimes, usage of reason codes, and abnormalbehavior of switching certificates from ”Revoked” to ”Good” status) capture ahighly heterogeneous landscape that lacks a revocation transparency standard.Finally, we argue for the deployment of such a standard and demonstrate theglobal impact of the mass revocation event, which took place during our measure-ment campaign. We compared the characteristics of the mass-revoked certificateswith the characteristics of other revoked and non-revoked certificates issued byLet’s Encrypt and the rest of the CAs, and found a limited number of biases,e.g., the biggest differences in the revocation rates depend on the origin CA, keytype, EV policy, and presence of embedded SCTs.
Acknowledgment:
This work was supported by the Wallenberg AI, Au-tonomous Systems and Software Program (WASP) funded by the Knut andAlice Wallenberg Foundation. eferences
1. Baseline Requirements for the issuance and management ofpublicly-trusted certificates, v1.7.2 (2020), https://cabforum.org/baseline-requirements-documents/
2. OneCRL (CA/Revocation Checking in Firefox) (2020), https://wiki.mozilla.org/CA:RevocationPlan
3. Apple: About upcoming limits on trusted certificates (2020), https://support.apple.com/en-us/HT211025
4. Chuat, L., Abdou, A., Sasse, R., Sprenger, C., Basin, D., Perrig, A.: SoK: Dele-gation and Revocation, the Missing links in the Web’s Chain of Trust. In: Proc.IEEE EuroS&P (2020)5. Chung, T., Lok, J., Chandrasekaran, B., Choffnes, D., Levin, D., Maggs, B.M.,Mislove, A., Rula, J., Sullivan, N., Wilson, C.: Is the Web ready for OCSP Must-Staple? In: Proc. IMC (2018)6. Cooper, D., Santesson, S., Farrell, S., Boeyen, S., Housley, R., Polk, W.: InternetX.509 Public Key Infrastructure certificate and Certificate Revocation List (CRL)profile. RFC 5280 (May 2008)7. Deacon, A., Hurst, R.: The Lightweight Online Certificate Status Protocol (OCSP)Profile for High-Volume Environments. RFC Editor, RFC 5019 (September 2007)8. DigiCert: DigiCert: Delay of revocation for EV audit inconsistency incident (2020), https://bugzilla.mozilla.org/show_bug.cgi?id=1651828
9. DigiCert: Inconsistent EV audits (2020), https://bugzilla.mozilla.org/show_bug.cgi?id=1650910
10. Durumeric, Z., Adrian, D., Mirian, A., Bailey, M., Halderman, J.A.: A searchengine backed by Internet-wide scanning. In: Proc. ACM CCS (2015)11. Google: CRLSets, https://dev.chromium.org/Home/chromium-security/crlsets , last accessed: September 202012. Google: Certificate lifetimes (2020), https://chromium.googlesource.com/chromium/src/+/master/net/docs/certificate_lifetimes.md
13. Google Trust Services: Certificate Policy v1.3, https://pki.goog/GTS-CP-1.3.pdf , OID = 1.3.6.1.4.1.11129.2.5.3; Last accessed: Jan. 21, 2021.14. Gustafsson, J., Overier, G., Arlitt, M., Carlsson, N.: A first look at the CT land-scape: Certificate Transparency logs in practice. In: Proc. PAM (Mar 2017)15. Internet Security Research Group (ISRG): Certification Practice Statement, Ver-sion 3.0 (Oct 2020), http://cps.letsencrypt.org , last accessed: Jan. 21, 2021.16. Kim, D., Kwon, B.J., Koz´ak, K., Gates, C., Dumitras, T.: The broken shield:Measuring revocation effectiveness in the Windows code-signing PKI. In: Proc.USENIX Security (Aug 2018)17. Koblitz, N.: Elliptic curve cryptosystems. Mathematics of computation (177),203–209 (1987)18. Korzhitskii, N., Carlsson, N.: Characterizing the root landscape of CertificateTransparency logs. In: Proc. IFIP Networking (June 2020)19. Korzhitskii, N., Carlsson, N.: Dataset for “Revocation Statuses on the In-ternet” PAM 2021 paper (2021),
20. Laurie, B., Langley, A., Kasper, E.: Certificate Transparency. RFC 6962 (2013)21. Let’s Encrypt: 2020.02.29 CAA Rechecking Bug (Mar 2020), https://community.letsencrypt.org/t/2020-02-29-caa-rechecking-bug/114591/3
2. Let’s Encrypt: Download affected certificate serials for 2020.02.29 CAA RecheckingIncident (Mar 2020), https://letsencrypt.org/caaproblem/
23. Liu, Y., Tome, W., Zhang, L., Choffnes, D., Levin, D., Maggs, B., Mislove, A.,Schulman, A., Wilson, C.: An end-to-end measurement of certificate revocation inthe Web’s PKI. In: Proc. IMC (2015)24. Mozilla: (2020), https://blog.mozilla.org/security/2020/07/09/reducing-tls-certificate-lifespans-to-398-days/
25. O’Brien, D.: Certificate Transparency Enforcement in Chrome and CTDay in London (2018), https://groups.google.com/a/chromium.org/d/msg/ct-policy/Qqr59r6yn1A/2t0bWblZBgAJ , last accessed: Jan. 202126. Rivest, R.L., Shamir, A., Adleman, L.: A method for obtaining digital signaturesand public-key cryptosystems. Communications of the ACM (2), 120–126 (1978)27. SANS Internet Storm Center: SSL CRL activity, https://isc.sans.edu/crls.html , last accessed: September 202028. Santesson, S., Myers, M., Ankney, R., Malpani, A., Galperin, S., Adams, C.: X.509Internet Public Key Infrastructure Online Certificate Status Protocol - OCSP. RFCEditor, RFC 6960 (June 2013)29. Santesson, S., Myers, M., Ankney, R., Malpani, A., Galperin, S., Adams, C.: X.509 internet public key infrastructure online certificate status protocol-ocsp. RFC6960 (2013)30. Scheitle, Q., Gasser, O., Nolte, T., Amann, J., Brent, L., Carle, G., Holz, R.,Schmidt, T.C., W¨ahlisch, M.: The rise of Certificate Transparency and its impli-cations on the Internet ecosystem. In: Proc. IMC (2018)31. Sectigo: Certificate search, https://crt.sh , last accessed: September 202032. Smith, T., Dickinson, L., Seamons, K.: Let’s Revoke: Scalable Global CertificateRevocation. In: Proc. NDSS (2020)33. Starfield Technologies, LLC: Certificate Policy and Certification Practice State-ment (CP/CPS), Version 4.9 (October 2020), http://certificates.godaddy.com/repository/ , last accessed: Jan. 21, 2021.34. Zhu, L., Amann, J., Heidemann, J.: Measuring the Latency and Pervasiveness ofTLS Certificate Revocation. In: Proc. PAM (2016) Appendix A. Other CA-based behavior comparisons
We have already seen that different CAs have different revocation-status-handlingpractices. To provide some additional insights, we obtained day-of-week distri-butions that capture when
CAs change the “Revoked” status to something else(Figure 9(a)); compare this to the distribution of the first certificate validityday (Figure 9(b)). Perhaps, the most noticeable are the weaker weekly patterns.While more than half of the CAs issue significantly fewer certificates with startdates during weekends (dark areas for Sat/Sun in Figure 9(b)), we did not ob-serve such weekly patterns for the revocation status changes. Instead, only a fewCAs have spikes of revocation status changes on a certain day (white squares inFigure 9(a)). For example, Starfield, GoDaddy (part of Starfield), and Digiden-tify update most of their statuses on Friday, and Japanese Registry on Sunday(Monday Japanese time). The distributions suggest that the relation betweenlast-status-change and certificate-validity-start days is not straightforward. Hav-ing said that, some of the CAs have even weekly distributions for both processes, onTueWedThuFriSatSun Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s (a) End of revocation status MonTueWedThuFriSatSun Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s (b) Start of certificate validity period Fig. 9.
Weekly distribution of certificate-validity-start day for the revoked certificatesand last-status-change day (from “Revoked” to something else).
MonTueWedThuFriSatSun Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s (a) Expiry day Le t ' s E n c r y p t G o D add y G oog l e A c t a li s na z w a . p l D i g i C e r t C O M O D O c P ane l S e c t i go S t a r fi e l d E n t r u s t G and i I n t e r ne t H e r ndon S w i ss S i gn G l oba l S i gn T E R E N AS y m an t e c Q uo V ad i s G eo T r u s t P l e x KP N SS L C o r po r a t i on D i g i den t i t y F N M T - RC M U SE R T RU S T J apan R eg i s t r y O t he r s (b) Expiry time Fig. 10.
Per-CA breakdown of expiry time of revoked certificates. which may suggest higher levels of automation (e.g., Let’s Encrypt, Google, Ac-talis, cPanel, Gandi, Herndon). Among the large CAs, DigiCert stands out withtheir pronounced weekly patterns for both processes. Similarly, there are somedifferences in the daily (Figure 10(a)) and hourly (Figure 10(b)) distributionsof the expiry times selected for certificates. Here, some of the large CAs (e.g.,Let’s Encrypt, GoDaddy, Google, GlobalSign) spread expiry times both acrossthe week and the hours of the days, whereas other large CAs (e.g., DigiCert, Co-modo, cPanel, Sectigo) always set certificates to expire at the same time of day.Although these differences may not have major security implications, perhaps,they demonstrate the lack of a standardized policy for managing the revocationstatus of expired certificates.