Evaluation of RxNorm in Ambulatory Electronic Prescribing

by Douglas S. Bell, Sean Michael O'Neill, Kerry Reynolds, Diane Schoeff

This Article

RAND Health Quarterly, 2011; 1(3):8


Drug prescriptions need to accurately identify the medications that prescribers intend for pharmacists to dispense. However, the set of computer-interpretable identifiers currently used in electronic prescribing (e-prescribing) does not support the specific needs of the prescribers and pharmacists who conduct e-prescribing transactions.

RxNorm, a drug nomenclature from the National Library of Medicine, assigns a rigorously derived, centrally maintained, and publicly available unique identifier to each clinically distinct drug. It therefore has the potential to greatly improve drug identification in e-prescribing transactions. Through laboratory research and pilot testing, the authors evaluated RxNorm's potential to improve how medications are represented in e-prescribing transactions. They found that RxNorm is highly complete for prescription drugs used in ambulatory practice that are within its scope and that it offers substantial efficiency and parsimony gains for communicating health plan formulary information to prescribers. The errors that arose during use were of low clinical significance. In live pilot testing, RxNorm identifiers were successfully added to prescription transactions sent between participating prescribers and pharmacies, and the pharmacies were able to use this information to check for dispensing errors or to disambiguate prescriptions. The authors found that RxNorm appears to provide drug identifiers that more accurately reflect the prescriber's intent than the drug identifier set currently used.

For more information, see RAND TR-941-CMS at https://www.rand.org/pubs/technical_reports/TR941.html

Full Text

All drug prescriptions need to accurately identify medications that the prescriber intends for the pharmacist to dispense. In e-prescribing, the use of unambiguous, computer-interpretable drug identifiers offers the possibility of computer assistance to prevent errors, both in prescribing and dispensing (Bell et al., 2004). Currently, the Food and Drug Administration's National Drug Code Directory is the standard source for computer-interpretable drug identifiers in e-prescribing transactions. However, the National Drug Code Directory has not been accurately maintained (Levinson, 2006). Furthermore, it was designed to distinguish among drug packages from different sources rather than to represent the intent of prescribers, who are rarely concerned about the drug source or packaging. Thus, the National Drug Code identifiers used in e-prescriptions sometimes misrepresent the prescriber's intent, potentially creating both new kinds of errors and the need for inefficient manual checks to ensure accurate interpretation.

The ideal system of drug identifiers for e-prescribing would have one unique identifier for every clinically distinct medication available, where clinically distinct refers to differences that matter when the drug is administered to a patient (as opposed to differences that matter in production and distribution). The identifier system would be complete if it included every drug that is currently available for prescribing, and it would be semantically precise if each clinically distinct medication were represented by only one identifier. In the context of e-prescribing, this semantic precision would enable each party to the prescription transaction to interpret without ambiguity the clinical drug that the prescriber intended. An ideal system of drug identifiers would also be easily managed by its users and have features that enable the maintenance of precise meaning as corrections are made and as the availability of drugs changes over time. Finally, it would cover other, more-abstract concepts. For example, it would represent the concept of a tablet that contains a particular ingredient (regardless of strength), and this would enable precise reference to this class of medications for decision support purposes, such as formulary checking or allergy checking.

RxNorm is a drug nomenclature from the National Library of Medicine (NLM) that is designed to fulfill the criteria outlined above by assigning a rigorously derived, centrally maintained, and publicly available unique identifier to each clinically distinct drug. Thus, RxNorm has the potential to greatly improve drug identification in e-prescribing transactions. In 2006, we found that RxNorm covered 99.0 percent of medications represented in a sample of 19,824 ambulatory e-prescriptions, but, for 5.2 percent of these prescriptions, independent users of RxNorm had not selected the same RxNorm concept to represent the prescription. About half of these disagreements were attributable to the existence of duplicate, clinically synonymous drug concepts in RxNorm. These duplications represent semantic errors because, as just noted, one goal of RxNorm is to have only one unique identifier for each clinically distinct drug concept. Since 2006, the NLM has expanded RxNorm's coverage by adding a new concept type for drug-device packages, and it has continued to improve its concept maintenance methods.

RxNorm could also be used to address technical problems that have impaired the functioning of the Formulary and Benefit (F&B) standard from the National Council for Prescription Drug Programs (NCPDP). The F&B information available to prescribers is often missing or inaccurate (Grossman et al., 2007; Wang et al., 2008), due in part to the challenge of maintaining sufficiently detailed and accurate files using National Drug Code numbers (NDCs). This is challenging because, for each prescribable drug, many F&B entries are needed to cover all of the NDCs that might be used (Bell et al., 2008).

The study described in this article evaluated RxNorm's potential to improve how medications are represented within the transactions that are currently used in ambulatory prescribing. These are the new prescription (NEWRX) transaction and the refill request (REFREQ) transactions within the NCPDP SCRIPT standard and the prescription drug insurance coverage information represented in the NCPDP F&B standard. In Part I of the study, we evaluated the use of RxNorm in a “laboratory” environment, using historical prescription data samples. In Part II, we evaluated the usability and interoperability of RxNorm within live prescriptions being transmitted between participating physician offices and pharmacies. We also evaluated the perceptions of participating pharmacists and prescribers regarding the medication management process both before and during the inclusion of RxNorm identifiers with transactions between pilot participants. However, RxNorm identifiers were added to new prescriptions after the medication had been selected, so RxNorm affected only pharmacy fulfillment, not the prescribing interface. (This decision enabled e-prescribing vendors to continue using their existing prescribing interfaces. They considered this important from a safety standpoint to ensure that user errors in drug selection would be no greater during the pilot than they are in general.)

Laboratory Testing

The Completeness and Reliability of RxNorm for e-Prescription Transmissions

Using a sample of 19,743 e-prescriptions, we estimated the coverage rate of RxNorm for representing clinical drugs, measured the six-month replacement rate of the RxNorm concepts used, assessed the consistency of two independent concept mappings, and investigated inconsistent mappings.

The April and October 2009 releases of RxNorm contained clinical drug concepts for all but one prescription in the sample (99.995 percent). Of the concepts used in the April release, 8.1 percent were superseded by new concepts in the October 2009 release. Two independent mappings produced different concepts for 676 e-prescriptions (3.4 percent), including differences in extended-release dose forms, salts, and metered-dose inhalers, but the differences had relatively low clinical significance.

Thus, RxNorm provides standardized concepts covering nearly all ambulatory e-prescriptions in a large sample derived mostly from primary care settings. The level of agreement among independent uses of RxNorm was relatively high, and the cases of disagreement had low clinical relevance. Nonetheless, mechanisms are needed to resolve potential ambiguities that remain in the use of some concept types, particularly extended-release dose forms and metered-dose inhalers. In this study, we propose an algorithm that could be used to identify potentially ambiguous extended-release concepts and to flag them for exclusion from use in e-prescribing.

The Efficiency of RxNorm in Representing Formulary and Benefit Information

Using the formulary status list (FSL) from a large pharmacy benefit manager, we automatically matched each FSL entry to its corresponding prescribable concept (i.e., Semantic Branded Drug, Branded Pack, Semantic Clinical Drug, or Generic Pack) and drug–dose form concept (i.e., Semantic Branded Dose Form or Semantic Clinical Dose Form). Drug–dose form concepts are concepts that aggregate all of the strengths for a given drug–dose form pairing. We then determined the extent to which the drug–dose form concepts, prescribable concepts, and NDCs could be used to most parsimoniously represent the set of formulary entries while still preserving the formulary status expressed for each entry.

We found that 52 percent of FSL entries (27,483 of 52,913) matched an RxNorm concept. Of those, 26,810 (97.6 percent) could be represented by an RxNorm concept instead of an NDC without loss of formulary status information. The other 673 entries (2.4 percent) were NDCs that represented different formulary statuses within the same RxNorm concept (e.g., ranitidine 150-mg tablets from some manufacturers are covered, and others are not). The 26,810 formulary entries could be represented by 8,911 prescribable RxNorm concepts. Thus, using RxNorm, the same formulary information could be expressed with about one-third the number of entries. If these could in turn be aggregated to higher-level drug–dose form concepts, file sizes could be further reduced to one-quarter the number of entries. Among the formulary entries that did not match to RxNorm, 67 percent were out of scope for RxNorm (one example is equipment), and the reminder represented NDC numbers that could not be interpreted using the available drug databases.

In conclusion, we found that RxNorm offers substantial efficiency and parsimony gains for formulary entries that are within its scope, potentially enabling greater accuracy through easier maintenance and greater simplicity of presentation. The 2.4 percent of entries that represented different formulary statuses within the same clinical drug concept should not need to be represented because the drugs represented are clinically equivalent and therefore interchangeable at the pharmacy. Trying to present these formulary differences to prescribers probably creates unnecessary complexity, especially in comparison with the very substantial inaccuracies that are currently prevalent in F&B information (Grossman et al., 2007; Wang et al., 2008). Thus, F&B files would be more usable and probably more effective if NDCs were completely replaced by RxNorm concept unique identifiers (RxCUIs). If RxNorm could be modified to include diabetes care supplies, it would substantially increase the proportion of F&B concepts that could be represented more parsimoniously in the F&B standard.

Live Pilot Testing

The Use of RxNorm in Live New Prescription and Refill Request Transactions

Five ambulatory physician practices (using two e-prescribing software systems), two retail pharmacies (both using the same pharmacy management software system), and one mail order pharmacy participated in our live pilot program. The software used at each site was modified to add RxNorm identifiers to outgoing transactions and to process RxNorm identifiers from incoming transactions. De-identified data from all transactions were collected for analysis. The retail pharmacy software vendor implemented an alert triggered when the RxCUI of the drug selected to represent an incoming prescription did not match the RxCUI received with the prescription. We also examined the potential for e-prescribing systems to use RxNorm in reconciling refill requests. At the mail order pharmacy, pharmacists flagged incoming new prescriptions if, based on the NDC and drug name string received, they considered prescriber's intent ambiguous. The pharmacist then judged whether the RxCUI received with the prescription was useful in disambiguating the prescription.

During a live pilot period of about 20 weeks, 3,829 new-prescription transactions were transmitted from participating prescriber offices to participating pharmacies. An RxCUI had been added to 96.3 percent of the 3,687 prescriptions that contained an NDC and for which complete data were retrieved. Among new prescriptions sent between partners that used different drug knowledge bases (e.g., sent from a MediSpan client to a First DataBank client), 4.5 percent (76 of 1,603) contained a representative NDC that could not be resolved in the pharmacy's drug knowledge base; 61 of these transactions (80 percent) had included an RxCUI, and, for 100 percent of these, the RxCUI accurately represented the drug prescribed. At the retail pharmacies, RxNorm mismatch alerts were recorded for 349 of 2,157 new prescriptions received (16 percent). Most of these were due to the fact that a pharmacy technician selected alternatives that would be expected to have a different RxCUI (e.g., he or she selected a tablet when the prescription was for a capsule or entered a branded concept, such as Keflex, when the prescription was for a generic concept, such as cephalexin). Some were due to differences in the versions of RxNorm used by the parties, including the use of some concepts that had been retired and one partner's choice not to use pack concepts.

Due to the short history of e-prescribing use among the prescribers involved, only two of the 893 refill request transactions conducted during the pilot could be mapped to a prior electronic prescription for the same patient within the previous ten months. Thus, there were insufficient historical data to test a proposed alert based on reconciling new refill requests with prior prescriptions.

At the mail order pharmacy (which used both MediSpan and First Databank to resolve NDCs), an RxCUI was sent with 1,444 of 1,495 prescriptions (96.6 percent). Of these, 28 (1.94 percent) were flagged as needing drug identity clarification, but the RxCUI received was considered helpful for only three of these (11 percent), due in part to the use of underspecified RxNorm terms, such as extended-release dosage forms that did not include a specific time interval.

In conclusion, we found that RxNorm could disambiguate most prescriptions when the representative NDC was not found in the recipient's drug knowledge base. However, use of RxNorm to alert for pharmacy data-entry errors resulted in a high rate of false alarms. This rate would have been lower had both parties regularly updated their RxNorm content and had pharmacy technicians regularly entered the precise drug name prescribed rather than making brand-name substitutions. RxNorm would be more useful for disambiguating prescriptions if underspecified terms, such as extended-release dose forms without a specific duration of action, were not used in prescriptions. RxNorm did not prove to be useful in reconciling refill requests because, at the time of the study, most patients had few prior e-prescriptions to be matched.

The Effects of RxNorm Use in Live Pilot Testing

Trained field researchers conducted site visits at each participating pharmacy and physician office at baseline and again after the switch to including RxNorm identifiers in transactions. Data were collected using multimethod observations and interviews. In addition to documenting changes that might be directly attributable to the use of RxNorm in the pilot, we also sought to elicit providers' views of the primary challenges associated with e-prescribing and drug identification.

At baseline, both prescribers and pharmacists perceived that, compared with handwritten prescriptions, e-prescribing systems often force prescriptions to be overspecified. Areas of overspecification included the drug itself (e.g., specifying a particular salt when other salts would be equivalent from the prescriber's standpoint), the dose form (e.g., specifying a capsule when a tablet would be fine), and brand-name versus generic drugs. Overspecification caused confusion among prescribers and required pharmacies to recontact prescribers in order to clarify intent. After RxNorm implementation, the intended RxNorm-based alerting feature was working for only one of the two retail pharmacies at the time of our site visit. Pharmacy technicians expressed annoyance at the alerts because most were false positives (i.e., the prescribed drug or its equivalent had in fact been accurately selected), but the pharmacists expressed satisfaction with the potential to prevent drug-selection errors.

In conclusion, live pilot testing revealed that RxNorm could potentially reduce the need for pharmacy callbacks to resolve ambiguous prescriptions, thereby improving efficiency both for prescribers and pharmacies. However, the problem of prescription overspecification will not be addressed as long as the National Drug Code remains the primary system for drug identification. RxNorm could improve pharmacies' ability to automatically check the accuracy of medication selections, either upon initial data entry (the use case we tested) or when a drug is selected from stock for dispensing.

The Readiness of RxNorm for Use in Ambulatory e-Prescribing

Overall, RxNorm appears ready for use to represent ambulatory prescriptions from primary care physicians. Its completeness was extremely high for the sample we examined, and unresolved synonyms in RxNorm were rare. Most important, errors in mapping NDCs to RxNorm, although not rare (they occurred in 3.4 percent of the sample), were of low clinical significance (e.g., involving minor differences in dose forms, salts, or inhaler canister sizes). Further, many of these errors would be preventable if potentially ambiguous dose forms and salts were flagged for exclusion from use in e-prescribing.

E-prescribing vendors and pharmacy vendors proved capable of adding accurate RxCUIs to the vast majority of prescriptions in a live pilot test, and RxCUIs had potential value in clarifying the 4.5 percent of prescriptions containing an NDC from one of the e-prescribing systems that could not be resolved in the pharmacy's drug knowledge base. However, RxNorm would have been more useful in accurately representing the prescriber's intent and in preventing calls for clarification if prescribers could have directly selected RxNorm concepts rather than products specified at the National Drug Code level. Overall, for prescribing, RxNorm appears to provide more-accurate drug identifiers than the National Drug Code. We recommend that NCPDP consider switching from the National Drug Code to RxNorm as the primary drug identifier in prescriptions.

RxNorm also appears ready to enable a substantially more parsimonious representation of the data in FSLs of the F&B standard. The formulary status distinctions that could not be reproduced with RxNorm represent supplier-level coverage differences that prescribers should not need to be concerned about. Thus, RxNorm offers promise of more-manageable FSLs, which could then be more easily expanded to represent the group-level variations in coverage that are currently a source of inaccuracy. However, given that the e-prescribing industry has not yet developed experience with using RxNorm in the F&B standard, live pilot testing of this approach is recommended.


Bell DS, Cretin S, Marken RS, Landman AB. A conceptual framework for evaluating outpatient electronic prescribing systems based on their functional capabilities. Journal of the American Medical Informatics Association. 2004;11(1):60–70.

Bell DS, Schueth AJ, Guinan JP, Wu S, Crosson JC. Evaluating the technical adequacy of electronic prescribing standards: Results of an expert panel process. Proceedings of the AMIA Annual Symposium. 2008:46–50.

Grossman JM, Gerland A, Reed MC, Fahlman C. Physicians' experiences using commercial e-prescribing systems. Health Aff (Millwood). 2007;26:w393–404.

Levinson DR. The Food and Drug Administration's National Drug Code Directory. Department of Health and Human Services, Office of Inspector General. 2006.

Wang CJ, Patel MH, Schueth AJ, et al. Perceptions of standards-based electronic prescribing systems as implemented in outpatient primary care: A physician survey. J Am Med Inform Assoc. 2009;16:493–502.

RAND Health Quarterly is produced by the RAND Corporation. ISSN 2162-8254.