Introducing Universal Evidence: A Search Engine for Research in Global Development

If you design, fund, or deliver development programs, you have to decide who to help, how to intervene, and where to invest limited resources. Finding relevant research to inform those decisions shouldn’t require searching across multiple research databases or guessing which terms researchers used. Universal Evidence is a search engine built to make that easier. It brings research from multiple databases into one search platform, helping you see what others tried — to address a given condition, with a given intervention, in a given place — and access relevant, published evidence that can guide your future work.

• Live app: https://www.universalevidence.com/
• Git repo: https://github.com/SteveHedden/universalevidence

The search for evidence in global development

Imagine you’re designing a program to reduce child malnutrition. You have a limited budget, a proposal deadline, and several plausible options: nutritional supplements, cash transfers, nutrition education, school meals, or support for local agriculture. Your team and community partners understand the local challenges. But you also want to learn from what others have tried. Which approaches have been studied with similar populations? In similar settings? What did those studies find, and how relevant are their findings to your program?

Answering these questions could be crucial as you choose an approach, explain a funding decision, or reconsider a program that isn’t delivering the results you expected. Research to support your pursuits likely already exists. And yet, given the limitations of today’s global research and publishing ecosystem, finding the relevant studies can take more time and specialist knowledge than your team has available.

Universal Evidence aims to make that discovery easier.

Closing the discovery gaps in global development research

Research on global development remains scattered across research databases and siloed within domains, and researchers often use different terms for closely related problems and interventions. A search for “malnutrition,” for example, should help you discover studies on stunting and wasting, but if the authors or publishers of these studies missed “malnutrition” as an explicit tag, potentially useful studies may be left out of your results.

You shouldn’t have to know every researcher’s terminology before you can find their work. That’s the gap that Universal Evidence (UE) aims to close. Backed by an extensive knowledge graph, our tool connects the terminology used across research repositories, so you can explore how others have already tried to solve a particular development-related problem.

It does not, by itself, tell you which approach will work in your local community. Instead, it surfaces evidence that might inform your own decisions as someone committed to on-the-ground change.

Example: The meaning of malnutrition

To illustrate the current gap further, let’s look at four studies on malnutrition. Each examines a different intervention, and each describes its conditions and interventions differently. Together, they show why relevant evidence can be difficult to find — and how connecting those descriptions makes it easier to discover.

The first study on the left is about treating a particularly severe and acute form of malnutrition (severe acute malnutrition or SAM) using therapeutic milk formula. But the intervention field doesn’t say “therapeutic milk formula”, it says “F75 milk feeds; F50 milk feeds; F35 milk feeds” which are types of milk formulas.

The second is about treating stunting (when a child is extremely short for their age due to malnutrition) using ready-to-use therapeutic food (RUTF). But the condition field doesn’t say “stunting” it says “Stunting prevention in children 6–59 months of age.” And the intervention field doesn’t say “Ready-to-use therapeutic food,” it says, “Lipid-based nutrient supplements / Ready to Use Supplementary Foods (RUSF)”.

The third study tests whether community health workers can reduce stunting and/or wasting (when a child is extremely underweight relative to their height) by encouraging caregivers to measure their child’s growth more frequently, giving nutritional counseling, and giving direct cash transfers for nutritious food. The condition field here doesn’t say stunting or wasting, but it does say “Height-for-age Z-score; Weight-for-height Z-score,” which are the ways stunting and wasting are measured, respectively.

Finally, the fourth study tests whether training farmers on agroecology practices, climate change, nutrition, and gender equality will increase dietary diversity. The intervention field here just says, “Mentor farmer training (Malawi)”.

This example includes a very small sample; there are millions of studies just like this, each published with its own approach (or lack of) to metadata. If we want to know the best way to reduce malnutrition and all we have is a million documents with this kind of loose metadata, we are fucked. A search for malnutrition won’t surface interventions focused on stunting if the authors wrote the calculation for stunting in the field. And even if it does say stunting in the body of the paper, we won’t see those results unless the query knows that stunting is a form of malnutrition.

The solution is to map these free text values to established concepts with established definitions, i.e., to move from strings to things. “Acute Malnutrition, Severe” is the same thing as “Severe acute malnutrition,” which is the same thing as “weight for height less than three standard deviations below the median.” When I search for interventions aimed at reducing the prevalence of children with stunting, I expect to see studies that have any of these values in their intervention fields. Further, these concepts must be structured hierarchically so that when I search for “malnutrition,” I see results for SAM, even if the SAM study isn’t tagged with “malnutrition.” The name for this structure of concepts is a taxonomy or a thesaurus.

Mapping values to concepts in a taxonomy or thesaurus is not easy. You can’t just match strings directly. An “exchange rate” can be of currencies or bodily fluids. “Irrigation” can be of cropland or mouths (in dentistry). “Discharge” can mean leaving a hospital, releasing wastewater, or, again, of bodily fluids. You get the idea: language is hard.

The visual above shows how part of the conditions and outcomes thesaurus is structured on the left and part of the interventions thesaurus is structured on the right. I define Malnutrition in the conditions and outcomes thesaurus as, “The persistent condition of inadequate or excess nutrition — encompassing both undernutrition and overnutrition — resulting from deficient, excessive, or imbalanced intake of nutrients or energy.” By that definition, SAM, stunting, and wasting are all types of malnutrition, while dietary diversity and child growth are not. With these thesauri in place, it’s possible to map the free text to these terms. The studies then connect the conditions on the left with the interventions on the right.

Now we can see which interventions have been tried to address different conditions; it enables us to build a knowledge graph (see image below) with thousands of entities and relationships. These four studies give us 15 nodes (six condition or outcome concepts and nine intervention concepts) and 22 connections between them.

So far, UE is a graph with nearly 10,000 nodes (7,000 states and 3,000 interventions) with about 70,000 evidence connections and approximately 51,000 supporting studies (see below). And that’s just by ingesting four data sources.

Visual representation of the entire Universal Evidence knowledge graph. Built using Gephi.

So back to the original question: What evidence exists for addressing malnutrition with different interventions?

If we look at Malnutrition itself (the broad term), then the intervention with the most evidence is combination drug therapy. This is because of all of the new obesity and weight management drugs like Ozempic. For treating severe acute malnutrition? UE shows the most evidence for ready-to-use therapeutic/supplementary food (RUTFs or RUSFs). And for stunting? Nutrition education. This makes sense. We have a lot of evidence for weight management drugs to treat obesity, RUTFs to treat an acute form of undernutrition, and education to treat a chronic form of undernutrition.

What better research discovery could mean for global development practitioners

What happens when it becomes easier to answer questions about the strength (or lack thereof) of evidence for a given intervention in addressing a given condition? Here are some ideas for things that can be built on top of UE:

1. Evidence summarization generator: Given a condition and a region, get a written brief about what’s been tried, who ran it, and any reported results, with citations to the original sources. Help readers check claims by tracing them to supporting studies and highlighting conflicting findings and limitations in the evidence.
2. Proposal grounding tool for funders and NGOs: Compare a draft project design with prior research, highlight relevant findings in the available evidence, and draft the evidence section of a funding application.
3. Systematic reviewer scaffolding tool: Help researchers discover relevant studies and organize their metadata, accelerating the search and screening stages of a systematic review.
4. Reconciliation API: Map an organization’s terminology to UE’s shared vocabulary, making its data easier to combine and compare.
5. Domain-specific AI: Use UE’s knowledge graph to train specialized models that reason about what works, for whom, and under what conditions, with answers grounded in traceable evidence. Researchers at Princeton have proposed this approach, using knowledge graphs to generate structured training material for specialized AI models. Their paper describes this as a path toward “domain-specific superintelligence.”
6. Evidence gap mapper: Show which combinations of conditions, interventions, and regions have substantial research and which have little discoverable evidence, helping funders prioritize new studies.
7. Research partner finder: Identify researchers and organizations that have studied or supported past research on a particular problem or delivered similar interventions, helping teams find collaborators or funders with relevant experience.

The principles behind Universal Evidence

Building things requires making decisions. Making decisions is easier with a set of principles. Below are the principles guiding Universal Evidence.

1. Driven by competency questions: No class, property, or vocabulary term is added unless it is needed to answer a competency question.
2. Three-tier architecture: Universal Evidence is organized as three distinct, interoperable layers: (1) an ontology defining what exists and how things relate, (2) controlled vocabularies cataloging instances of those classes, and (3) observational data providing evidence connecting instances. This separation — theory, instances, evidence — is what allows knowledge to accumulate, evolve, and be reasoned over as understanding improves. It is the structural model that made healthcare the global leader in knowledge graph maturity.
3. Modular by design: Modularity operates at two levels. At the system level, each tier (ontology, vocabularies, evidence) is independently useful. The controlled vocabularies (thesauri) are public goods that external systems can adopt without using the query product. The metadata about the data sources themselves are also modular, so we can add sources over time. This independence requires a hard constraint: no cross-vocabulary assertions. Cross-vocabulary relationships belong in the observations layer or are expressed through Simple Knowledge Organization System (SKOS) alignment properties only.
4. Open source: Published under Apache 2.0 license, Universal Evidence is free for anyone to use, extend, or build on. The thesauri are also published separately and open source licensed.
5. Reuse existing ontologies and vocabularies: No new terms may be coined unless no suitable existing term exists. Map every class and property to an established vocab before defining anything new.
6. W3C standards: Universe Evidence uses established web standards: Resource Description Framework (RDF), RDF Schema (RDFS), Web Ontology Language (OWL), Simple Knowledge Organization System (SKOS), Shapes Constraint Language (SHACL), PROV-O, Dublin Core Metadata Terms (DCTERMS), Data Catalog Vocabulary (DCAT), as well as SPARQL for querying, TTL as the authoring serialization, and JSON-LD for consumption.
7. Provenance first: Every assertion in the graph carries a source.
8. Pre-competitive infrastructure: The ontology and controlled vocabularies are shared public goods. Development organizations should compete on analysis and programming, not on the definition of terms.
9. Build incrementally: Start narrow and deep, allowing ontology to evolve as understanding improves — not top-down from a complete model.
10. Materialize only what has no API: Universal Evidence stores study metadata only for sources with no machine-readable interface. For sources with maintained APIs, the platform provides vocabulary alignment only and queries live. The goal is integration, not duplication. The query interface for sources need not be SPARQL. It may be any API that accepts queries in Universal Evidence terms, fans out to the appropriate sources, and returns a unified response.
11. Associative, not causal: Following the model of SNOMED CT and MedDRA, observations record that a condition or outcome was *associated with* an intervention in a given context. Causality is never asserted in the base layer. Causal interpretation is the work of analysts querying the graph, not a property stored in it. This keeps the graph usable across theoretical traditions that disagree on causal mechanisms.
12. Theory-neutral: The ontology does not privilege any development theory. The capabilities approach, institutional economics, structural transformation, and empirical traditions are all valid lenses; each can contribute observations and query the graph on its own terms. No class or property encodes a causal or normative commitment to one tradition over another.

Building connections with our three-tier architecture

In accordance with the second principle above, UE follows a three-tier architecture. Below is one visual description:

Another way to look at the architecture is as concentric rings. I like thinking of knowledge graphs this way because it helps to illustrate how a few simple concepts can connect entire universes of information. The core ontology has just eight key concepts, which I’ll define below (Subject, State, Intervention, Indicator, Outcome Measurement, Region, Evidence, and Crosswalk Entry). The next layer out contains the controlled vocabularies. There are two core thesauri (States and Interventions) and one taxonomy (Regions). Crosswalks then match each of the concepts in the core controlled vocabularies to the free text fields in the sources. The size of the crosswalks (almost 200,000 connections) is more than 10 times the number of concepts in our controlled vocabularies.

That also points to the benefit of using controlled vocabularies in the first place: they drastically reduce the number and complexity of crosswalk entries required for data alignment. Take ClinicalTrials.gov for example. There are about 226,000 studies relevant to Universal Evidence that we want to retrieve. They use the Medical Subject Headings (MeSH) vocabulary for their condition field. So mapping conditions terms from ClinicalTrials.gov to our conditions thesaurus is relatively easy. We just need to find the MeSH term (which has a description) that best aligns with a concept in our thesaurus. There are less than 5,000 unique MeSH terms in the condition field of ClinicalTrials.gov for the studies we are interested in.

They also use MeSH in the intervention field, but coverage is not as good. About half the studies we are interested in have no MeSH term in the intervention field — it is free text. There are over 212,000 unique free text strings in the intervention field for the same collection of studies. That means we need 212,000 crosswalk entities mapping these 212,000 strings to concepts in our intervention thesaurus if we want to retrieve and aggregate them. A real pain in the ass. I have only gotten through about 9K of them so far.

Tier 1: Ontology

An ontology is the theory underpinning the knowledge graph. It defines what exists (classes) and how those classes interact with each other. In medicine, ontologies developed over time as scientists made discoveries. In the beginning, we used plants to treat ailments. The ontology had two classes: herbal remedies and ailments, and one relationship: treats. When scientists discovered diseases could be caused by bacterium, the ontology grew to include pathogens. When we found things that could kill bacteria, we included microbes in the ontology.

Global development is not a hard science, and the classes in a development ontology are not as easy to verify. There are definitely theories underpinning development, however.

I have applied principles 1 and 5 above: “Be driven by competency questions” and “Reuse existing ontologies and vocabularies.” The competency question that will drive the entire first version of UE is: What evidence exists for addressing [a given condition] with [a given intervention] in [a given region]?

This question is very similar to the kinds of questions healthcare professionals ask, which is why I stole an ontology from them. There is an existing ontology that was built for describing healthcare evidence that we can repurpose for describing Universal Evidence: the PICO ontology. This ontology is a formalized version of the PICO Process, a mnemonic device used in evidence-based medicine to frame and answer a clinical or healthcare-related question. PICO stands for Population (or sometimes patient or problem), Intervention, Comparison (or sometimes control), and Outcome. For example, “In children with headache, is paracetamol more effective than placebo against pain?”

The PICO ontology has, not surprisingly, key classes of Population, Intervention, Comparison, and Outcome. In their ontology, healthcare professionals treat a Population that has a condition. The Population has properties like Age, Sex, and Condition. “Female children with malnutrition” is an instance of a Population. UE surfaces evidence for conditions of things other than populations. We are not just interested in how interventions affect populations but also other kinds of things like the economy and the environment. To accommodate this expansion, rather than use Population, UE uses Subject, which can be a population but can also be a natural or human-made system.

When trying to list out all the different conditions, I realized that they are very similar to all of the outcomes. The same concept can play one of two roles in a study: the Condition (something a study selects subjects on, e.g. “children with Malnutrition”) or an Outcome (something a study measures as a result, e.g., “change in Malnutrition rate”). These are the same underlying concept playing two different relational roles. So I deliberately put both together, under a class called “State”. These states can be measured by Indicators (another class). And an Outcome measurement is the observation of an indicator for a given piece of evidence.

This ontology is described visually below:

There is a class called “Evidence” that is crucial to the whole project. The reason the lines between that class and the others are dotted is because there is NO relationship between the CLASS of evidence and any of the other classes. Each instance of evidence, i.e., an individual study, is connected to a state and an intervention and, potentially, an outcome, but the class of evidence is not connected to anything. Likewise, there is no ontological connection between interventions and states. This is by design: the only way an intervention can be related to an outcome is through EVIDENCE.

Here are the explicit definitions of these classes:

State: A state of wellbeing, development, or a social, ecological, economic, institutional, or health system. The same state may play a condition role or an outcome topic role in a study.

Get Steve Hedden’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Intervention: A deliberate action with a stated development objective; something an actor does with the intent to change a condition. Examples: cash transfer, teacher training, vaccine distribution, information campaign.

Subject: The bearer of a state — the entity that can be in a problematic or noteworthy condition. Covers human populations (at any scale of aggregation), human-made systems, and natural systems. Corresponds loosely to the P (Population) in PICO but scoped to bearer types rather than study populations.

Evidence: A record of a study, trial, or evaluation drawn from a research registry or repository. Carries axis tags (ue:condition, ue:intervention, ue:outcome, ue:location) linking it to Tier 2 vocabulary concepts. Descriptive metadata (title, abstract, date, identifier, publisher) uses dcterms; provenance is inherited from prov:Entity.

Region: A geographic entity — country, sub-national region, or multi-country grouping. Country-level instances align with ISO 3166 / UN M.49.

Indicator: A standardized, operationalized metric used to observe or track a state.

Outcome measurement: A thin per-study measurement node that links an evidence record to the state measured as an outcome and carries only per-study outcome metadata such as direction, intent, and status.

Crosswalk entry (not pictured above): A source-text-to-taxonomy mapping used to normalize source vocabulary or free-text fields against UE vocabulary concepts.

Universal Evidence’s ontology also includes SHACL constraints: machine-readable rules that check whether data follows the model. These require core concepts to have a preferred label and belong to a vocabulary, prevent duplicate preferred labels within a thesaurus, and keep State and Intervention hierarchies separate, with at most one broader parent per concept. Indicators must identify the States they measure rather than sit inside those hierarchies. Evidence records must identify their source and study ID, while outcome measurements must identify a measured State. The constraints distinguish errors from warnings, allowing missing definitions or recommended metadata to be flagged for review. Together, these checks help keep the graph consistent and traceable; they validate its structure.

The full ontology is here. In addition to PICO, the UE ontology uses established web standards: Resource Description Framework (RDF), RDF Schema (RDFS), Web Ontology Language (OWL), Simple Knowledge Organization System (SKOS), Shapes Constraint Language (SHACL), PROV-O, Dublin Core Metadata Terms (DCTERMS), Data Catalog Vocabulary (DCAT), as well as SustainGraph and Basic Formal Ontology (BFO).

Tier 2: Controlled vocabularies

The second tier is about the controlled vocabularies. In addition to the core thesauri, tier 2 contains a regions taxonomy, metadata for the sources (evidence repositories) as well as crosswalks between those sources and our thesauri. The goal of this second tier is two-fold: to create thesauri to align instances of each of the classes across data sources and to document how to get the data from each of the sources. Let’s go through them one at a time.

But first, a note on the distinction between taxonomies and thesauri. Following Heather Hedden (no relation) and Jessica Talisman, I use taxonomy to mean a hierarchical organization of concepts, and thesaurus for a controlled vocabulary that also includes synonym equivalence and associative relationships, such as “related” concepts. Our states (conditions and outcomes) and interventions thesauri include broader, narrower, and related relationships; preferred labels and synonyms; definitions and scope notes; and mappings to external vocabularies. I’ve extended these with domain-specific classes and relationships defined in our ontology. Our regions vocabulary, meanwhile, organizes geographical places hierarchically, using GeoNames identifiers alongside our own regional groupings. I therefore refer to it as a taxonomy.

Thesauri / Taxonomies

The core of tier two are the thesauri: one for states (which includes indicators and subjects) and one for interventions. Creating these was incredibly time consuming — this was by far the hardest part of this whole project expect maybe the deployment architecture. Each of these thesauri (and the UE ontology) has its own github repo. They are each standalone products that can be used by others for other applications.

1. Universal Evidence Ontology
2. Universal Evidence Conditions and Outcomes Thesaurus
3. Universal Evidence Interventions Thesaurus

The way I made these thesauri is that I started with the data sources themselves. Some of these sources have well defined terms in the fields I mapped to in the metadata step above, but some just have free text. There are also existing vocabularies out there that were useful guides. Here are the existing vocabularies, classifications, and registry terminology that informed UE:

1. States: Medical Subject Headings (MeSH), UN Thesaurus, keywords extracted from the American Economic Association (AEA) registry for randomized controlled trials.
2. Interventions: Intervention descriptions from the AEA RCT registry, ClinicalTrials.gov, ISRCTN, the Food and Agricultural Organization (FAO) of the UN’s vocabulary AGROVOC, and the OECD’s purpose codes.
3. Indicators: The UN’s Sustainable Development Goals (SDGs) indicator framework and the World Bank’s World Development Indicators (WDI).
4. Regions: GeoNames and World Bank regions.

We have a lot of the ingredients now to create thesauri:

1. An ontology that defines each of the classes, relationships between them and constraints on how the vocabularies are structured.
2. Mirrors of each term used by each source from which we want to retrieve data.
3. Best practices and guidance on how to create thesauri. Big shoutout to Heather Hedden and her book The Accidental Taxonomist for explaining exactly how to build these things.

Using these ingredients, I created skills for agents to build and maintain the thesauri. That will be the subject of a future post, but there are four main roles: expansion agents map source terminology to existing concepts and add new concepts or synonyms where needed; scaffolding and restructuring agents build the hierarchy and reorganize branches when needed; auditing agents flag potential problems, such as duplicates, questionable parent–child relationships, or incorrect mappings; and hygiene agents investigate those findings against the source material, decide whether a change is warranted, and apply and validate the corrections.

The UE thesauri and taxonomies use Resource Description Framework (RDF), and Simple Knowledge Organization System (SKOS) to represent concepts, labels, definitions, hierarchies, and mappings. Their construction is informed by ISO 25964 and ANSI/NISO Z39.19, with SHACL constraints checking consistency against the UE ontology.

Source metadata

The data sources I used for this project are: The American Economic Association’s (AEA) registry for randomized controlled trials; ClinicalTrials.gov, the WHO International Clinical Trials Registry Platform, and the UK’s clinical study registry ISRCTN Registry. One key vocabulary in tier 2 is metadata about the sources themselves. This is crucial for two reasons:

1. Some of these sources have APIs and some do not. According to principle 10, we should “materialize only what has no API.” This means that if the source has an API, we should use it, rather than duplicating all of that metadata locally. This metadata file thus needs to contain information on the API endpoint (where it is, how to query, what format is returned, rate limits).
2. Each source has different names for their fields. One source may contain a field called “keywords,” which we can use to align with what we call “interventions.” The mapping of the source-specific field names to the UE ontology lives in this metadata vocab.

These mappings from source-specific field names to our ontology serves two roles: 1) when a user asks a question, this mapping translates it into a query against that source, and 2) when results from the source are returned, that mapping guides how to aggregate the results by class.

I used Dublin Core Metadata Terms (DCTERMS) to describe each source’s title, publisher, license, and other descriptive metadata, alongside Data Catalog Vocabulary (DCAT) to identify datasets, data services, and API endpoints. UE-specific properties describe how each source is accessed and how its fields map to the shared search axes, making this vocabulary both a source catalog and a machine-readable guide for integrating the data.

Mirrors

Rather than try to map their terms directly onto the core UE thesauri (and blow our thesauri up with hundreds of thousands of synonyms) I created “mirrors” of each of the relevant fields from each of the sources. This serves the dual role of 1) acting as one end of the bridge between the thesauri and the source data and 2) providing input into what should be in the thesauri in the first place.

The mirrors use Resource Description Framework (RDF), with Simple Knowledge Organization System (SKOS) to organize source terms and Dublin Core Metadata Terms (DCTERMS) to record source attribution where applicable.

Crosswalks

The final piece of tier two is the vocabularies that connect the thesauri to the mirrors. The agents described above can help with this quite a bit. As they go through all of the terms in a given field from a source they have to decide if it should be added to a thesauri and then update a crosswalk so that the mirror terms maps to that thesauri term.

The crosswalks also use RDF and SKOS, with relationships such as skos:exactMatch, skos:closeMatch, and skos:relatedMatch distinguishing equivalent, approximately equivalent, and related concepts. UE-specific properties preserve the original source text and mapping metadata.

Tier 3: Data sources

The third tier of the architecture is the evidence itself. The evidence is what drives the entire structure of this graph — this is what allows us to make connections between a given intervention and a condition. For the first version of UE, I have focused on the following four sources:

AEA RCT Registry

The American Economic Association’s registry records randomized controlled trials in economics and other social sciences. It documents study designs and provides links to supporting materials, such as research protocols, survey instruments, and findings.

This contains a ton of data (nearly 12,000 trials) and is economics focused so covers things other than health (other sources are health focused). They tag each trial with keywords. They also have a metadata field called “intervention” and a field called “primaryOutcome”. The intervention field contains free text, however. This study about malnutrition in India has this in the intervention, “We will provide mothers with children aged 6–23 months with complementary feeding kits. In addition, we will also provide IVR-based interventions.” The primary outcome field is also long, freeform text but it is also really more of a goal — this is a registry which means this is data that was recorded when the trial started. Our definition of outcome requires that it be recorded AFTER the intervention was implemented. For region, they have a field called “country,” where they put plain text names of the country where an intervention occurred.

AEA does not have an API, so we have to materialize all of this metadata locally in UE.

ClinicalTrials.gov

ClinicalTrials.gov is a public database of clinical research studies maintained by the U.S. National Library of Medicine, with studies conducted worldwide. Records describe interventional and observational studies, including their objectives, eligibility criteria, locations, and results when submitted.

ClinicalTrials.gov has a field for condition which contains values from the Medical Subject Headings (MeSH) thesaurus. That is a proper controlled vocabulary that makes mapping and querying a lot easier. ClinicalTrials.gov also has fields for intervention which they call “interventions” and outcomes, which they call “primaryOutcomes”. ClinicalTrials.gov also automatically assigns MeSH codes to interventions but only about half of the studies we are interested in have MeSH terms in their interventions fields. They use free text for the other half and for the outcomes field. They do use succinct terms at least for outcomes and not complete sentences. ClinicalTrials.gov has a field called contactsLocationsModule.locations[].country where they put free text country names also.

ClinicalTrials.gov has an API that we can query at runtime so we don’t have to store all of the data locally.

ISRCTN

ISRCTN is a clinical study registry that assigns unique identifiers and publishes information about planned, ongoing, and completed research. Originally focused on randomized controlled trials, it now includes a broader range of health studies, including observational research.

For condition, it has a field called conditionCategory which uses a controlled vocabulary of 23 categories — things like “Nutritional and Metabolic Diseases” or “Infections and Infestations.” This is a proper controlled vocabulary, though a coarse one, and it is health focused. For conditions outside health — governance, agriculture, education — ISRCTN falls back to a free-text condition field. Intervention is free text in a field called intervention. Outcomes are pre-specified measures in a field called outcomeMeasures — more structured than AEA but still not a controlled vocabulary. For region, ISRCTN has a field called recruitmentCountry with plain text country names.

ISRCTN also has an API that we query at runtime.

WHO ICTRP

The World Health Organization’s International Clinical Trials Registry Platform brings together trial records from registries around the world. Its search portal provides a single access point for information about ongoing and completed clinical trials.

The WHO International Clinical Trials Registry Platform is a meta-registry — it aggregates trial registrations from 18 contributing registries around the world, including registries in China, India, and Africa that are not captured by AEA or ClinicalTrials.gov. This is its main value: geographic coverage. The fields are called Condition, Intervention, Primary outcome, and Countries — all free text, all heterogeneous, because the contributing registries each have their own formats. There is no controlled vocabulary for any axis. Completeness varies significantly by contributing registry.

We access it not via an API but by manual CSV export per condition search, which is a significant practical limitation.

The first step towards sustainable data infrastructure for global development

Universal Evidence is an evidence search engine. It is a way to find research that supports or challenges claims. It is built for people designing, funding, and delivering programs to improve lives. It helps them test claims and make better-informed decisions, based on evidence over opinion.

Universal Evidence is the culmination of years of work and deep thinking on these issues. It is the first step towards several goals: a sustainable data infrastructure for global development; a reusable architecture for building similar infrastructures in other domains; and the most idealistic — a way of guiding decisions based on evidence rather than opinions. There is a lot more work to do to achieve these goals.

Next steps

Here are some of my ideas for next steps, not necessarily in order of priority:

1. Expand the number of sources we are integrating. There are a lot of databases out there containing a lot of potentially useful evidence.
2. Improve thesauri. As I mentioned earlier, this was probably the hardest part of this whole project, and I had to build a significant AI infrastructure to help me do it. There is still work to do, however.
3. Perform meta-analysis on the graph. Now that we have hundreds of thousands of pieces of evidence connecting thousands of concepts, we can look for patterns. What are the most prevalent interventions? How much evidence exists for addressing Malaria with Insecticide-treated nets? A lot more than by treating it with Insect control.
4. Build prototypes. Some of the things listed above, like an evidence summarization generator, can be built immediately and evolve with Universal Evidence over time.
5. Expand ontology to answer more competency questions. For example, what are the most severe development challenges in a given region today? Which populations are most impacted by each condition? What were the outcomes of interventions implemented to address a given condition? In what contexts do similar interventions succeed or fail? Where do sources disagree or express uncertainty? What trade-offs exist between addressing different conditions? What major knowledge gaps limit decision-making in this region?
6. Standardize how data is shared. We should collectively encourage more of these registries to use controlled vocabularies when tagging their data.
7. Convert more research into RDF. Turn existing resources that are NOT yet in formal RDF into actual taxonomies, ontologies, or thesauri for ease of use.

About the author

I spent the first chunk of my career in international development. I worked at an academic research center at the University of Denver, at a think tank in South Africa, and at multiple tech companies. I’ve used quantitative models to make long term forecasts on demographics, economics, water scarcity, and energy. I’ve used geographic information systems (GIS) and predictive models to estimate food insecurity and conflict in sub-Saharan Africa and Southeast Asia. I’ve used graph analytics to analyze relationships between business people and sanctioned companies.

I spent the second half of my career helping large companies manage their data, primarily in the healthcare and finance domains. Using knowledge graphs and semantic web technologies, these large companies can query across systems to discover and design products faster, determine potential adverse effects of product, and give better medical or financial advice.

I am now combining what I’ve learned about how large companies manage their data, and how healthcare and finance manage their domain data, with the problems I’ve experienced in development. The same problems that knowledge graphs solve in medicine and finance can be directly applied to development. Universal Evidence is my attempt at showing how this can be done.