Skip to content

Refactor the shape of the doc key in the REST API#3170

Merged
lintool merged 11 commits intomasterfrom
issue-3169
Mar 24, 2026
Merged

Refactor the shape of the doc key in the REST API#3170
lintool merged 11 commits intomasterfrom
issue-3169

Conversation

@lintool
Copy link
Copy Markdown
Member

@lintool lintool commented Mar 23, 2026

PR to address #3169 - pointed Codex at the issue and asked it to implement... wasn't able to one-shot but had to iterate a bit.

Initial patch, will circle around and refactor, de-slopify, etc.

@lintool lintool requested review from ronakice and sahel-sh March 23, 2026 20:22
@lintool lintool marked this pull request as draft March 23, 2026 20:22
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a7638d1061

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 23, 2026

Codecov Report

❌ Patch coverage is 75.49020% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.66%. Comparing base (be322a3) to head (1535726).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/main/java/io/anserini/api/RestServer.java 76.74% 6 Missing and 4 partials ⚠️
...in/java/io/anserini/util/PrebuiltIndexHandler.java 60.00% 4 Missing and 4 partials ⚠️
src/main/java/io/anserini/cli/CliUtils.java 76.00% 3 Missing and 3 partials ⚠️
src/main/java/io/anserini/cli/Search.java 85.71% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3170      +/-   ##
============================================
+ Coverage     74.60%   74.66%   +0.06%     
- Complexity     1796     1822      +26     
============================================
  Files           201      201              
  Lines         13631    13684      +53     
  Branches       1793     1804      +11     
============================================
+ Hits          10169    10217      +48     
- Misses         2734     2738       +4     
- Partials        728      729       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@lintool
Copy link
Copy Markdown
Member Author

lintool commented Mar 24, 2026

I've unified the shape the doc in REST API, Search CLI, and ExtractQueriesAndDocumentsFromTrecRun CLI.

Here's what it currently looks like: first start REST endpoint:

bin/run.sh io.anserini.api.RestServer --port 8081

Output:

% curl "http://localhost:8081/v1/msmarco-v1-passage/search?query=what%20is%20anserini&hits=5" | jq
{
  "api": "v1",
  "index": "msmarco-v1-passage",
  "query": {
    "text": "what is anserini"
  },
  "candidates": [
    {
      "docid": "2161721",
      "score": 10.3433,
      "rank": 1,
      "doc": "Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural..."
    },
    {
      "docid": "2161717",
      "score": 10.0202,
      "rank": 2,
      "doc": "Answer Wiki. Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural of genus) contain 17 species, which I will discuss here."
    },
    {
      "docid": "5246850",
      "score": 9.5717,
      "rank": 3,
      "doc": "The sounds of a very common farm bird, the domestic goose! Geese are waterfowl belonging to the tribe Anserini of the family Anatidae.This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese). A number of other birds, mostly related to the shelducks, have goose as part of their name.eese are waterfowl belonging to the tribe Anserini of the family Anatidae. This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese)."
    },
    {
      "docid": "5246852",
      "score": 7.0828,
      "rank": 4,
      "doc": "For other uses, see Goose (disambiguation) . Geese are waterfowl belonging to the tribe Anserini of the family Anatidae. This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese).A number of other birds, mostly related to the shelducks, have goose as part of their names.his tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese). A number of other birds, mostly related to the shelducks, have goose as part of their names."
    },
    {
      "docid": "2737045",
      "score": 2.6595,
      "rank": 5,
      "doc": "whats good; whats gooday [whats good] B; What's good boys? whats good family? what's good for the geese? whats good in the hood; What's good mo; What's good nigga; What's good wit chu; what's good with the butt? Whats good wit it; whats goody; What's goodybags? What's Goy? What's Grapes? What's Gravity; What's gravy; whats gravy in the navy; what's growin' long; whats guanin"
    }
  ]
}

% curl "http://localhost:8081/v1/msmarco-v1-passage/search?query=what%20is%20anserini&hits=5&parse=false" | jq
{
  "api": "v1",
  "index": "msmarco-v1-passage",
  "query": {
    "text": "what is anserini"
  },
  "candidates": [
    {
      "docid": "2161721",
      "score": 10.3433,
      "rank": 1,
      "doc": "{\n  \"id\" : \"2161721\",\n  \"contents\" : \"Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural...\"\n}"
    },
    {
      "docid": "2161717",
      "score": 10.0202,
      "rank": 2,
      "doc": "{\n  \"id\" : \"2161717\",\n  \"contents\" : \"Answer Wiki. Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural of genus) contain 17 species, which I will discuss here.\"\n}"
    },
    {
      "docid": "5246850",
      "score": 9.5717,
      "rank": 3,
      "doc": "{\n  \"id\" : \"5246850\",\n  \"contents\" : \"The sounds of a very common farm bird, the domestic goose! Geese are waterfowl belonging to the tribe Anserini of the family Anatidae.This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese). A number of other birds, mostly related to the shelducks, have goose as part of their name.eese are waterfowl belonging to the tribe Anserini of the family Anatidae. This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese).\"\n}"
    },
    {
      "docid": "5246852",
      "score": 7.0828,
      "rank": 4,
      "doc": "{\n  \"id\" : \"5246852\",\n  \"contents\" : \"For other uses, see Goose (disambiguation) . Geese are waterfowl belonging to the tribe Anserini of the family Anatidae. This tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese).A number of other birds, mostly related to the shelducks, have goose as part of their names.his tribe comprises the genera Anser (the grey geese), Branta (the black geese) and Chen (the white geese). A number of other birds, mostly related to the shelducks, have goose as part of their names.\"\n}"
    },
    {
      "docid": "2737045",
      "score": 2.6595,
      "rank": 5,
      "doc": "{\n  \"id\" : \"2737045\",\n  \"contents\" : \"whats good; whats gooday [whats good] B; What's good boys? whats good family? what's good for the geese? whats good in the hood; What's good mo; What's good nigga; What's good wit chu; what's good with the butt? Whats good wit it; whats goody; What's goodybags? What's Goy? What's Grapes? What's Gravity; What's gravy; whats gravy in the navy; what's growin' long; whats guanin\"\n}"
    }
  ]
}

% curl "http://localhost:8081/v1/msmarco-v1-passage/doc/2161721" | jq
{
  "api": "v1",
  "index": "msmarco-v1-passage",
  "docid": "2161721",
  "doc": "Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural..."
}

% curl "http://localhost:8081/v1/msmarco-v1-passage/doc/2161721?parse=false" | jq
{
  "api": "v1",
  "index": "msmarco-v1-passage",
  "docid": "2161721",
  "doc": "{\n  \"id\" : \"2161721\",\n  \"contents\" : \"Geese are members of the tribe Anserini in the family Anatidae, which also includes ducks. Anserini contains three (sometimes two to five, depending on who you ask) genera: Anser, Chen, and Branta. Together, these three genera (which is the plural...\"\n}"
}

Note the parse flag.

@lintool
Copy link
Copy Markdown
Member Author

lintool commented Mar 24, 2026

% curl "http://localhost:8081/v1/msmarco-v2.1-doc-segmented/search?query=what%20causes%20climate%20change&hits=5" | jq
{
  "api": "v1",
  "index": "msmarco-v2.1-doc-segmented",
  "query": {
    "text": "what causes climate change"
  },
  "candidates": [
    {
      "docid": "msmarco_v2.1_doc_05_273042196#0_275558320",
      "score": 9.8328,
      "rank": 1,
      "doc": {
        "url": "http://www.climatechangechallenge.org/Resource%20Centre/Climate-Change/3-what_causes_climate_change.htm",
        "title": "Causes of Climate Change",
        "headings": "Page Contents - Causes of Climate Change\nPage Contents - Causes of Climate Change",
        "segment": "Causes of Climate Change\n\n\n\n\n\n\n\n\n\n\n\n\nthe\nimpact and effects of climate change\nwhat is\nclimate change\nwhat are greenhouse gases\ncauses\nof climate change\nwhat is\ncarbon footprint\nwhat\naction can we take\nfeatured articles\nHow recycling mobile phones can benefit the environment\nGlobal warming, fact or fable\nHow energy companies rip you off\nClimate change: a summary of the science by The Royal Society\nBellamy, Climate Change not Man Made\nCombating climate change- China’s contribution to the expansion of Africa’s renewable energy sector\nThe Climate Change Challenge for British Woodland\nDo Volcanoes cause climate change\nDisposable Nappies (diapers) - No Worse for the Environment Than Cloth Nappies\nWalking to the shops damages planet more than going by car\nCauses of Climate Change slammed by 140 Scientists\n86 million Americans without healthcare\nThe truth about climate change\nRenewable energy from biomass and biofuel\nThe 10 big energy myths\nThe electric car\nLessons from the Copenhagen Climate Change Conference\nPage Contents - Causes of Climate Change\nCauses of Climate Change\nWhat Causes Climate Change in detail\nMain Contributors and Causes of Climate Change\nAgriculture as a Contributor to the Causes of Climate Change\nDeforestation as a contributor to the Causes of Climate change\nIncrease in global temperatures\nShattering the Green myths\nCauses and Effects of Climate Change in Ethiopia\nExisting Work and Knowledge regarding climate change\nThe Royal Society's 2010 Report looks at the science behind the causes of Climate Change\nNitrous Oxide No Laughing Matter\nScientific Basis of Climate Change and Causes\nClimate Change and the IPCC (Intergovernmental Panel on Climate Change)\nNASA's Earth Observation System monitoring the changing climate\nLivestock and Climate Change Causes\nEmotional Causes of Climate Change Denial\nUnited States Global Research Climate Program - \"human activities have increased additional heat - trapping gases\"\nCauses of climate change millions of years ago\nBlack carbon now second largest cause of climate change and key human contributor to global warming\nGreedy Lying Bastards - a powerful film about the people who cast doubts on the causes of climate change\nJoint science academies’ statement on the Global response to climate change and causes\nCauses of Climate Change\nThe causes of climate change can be divided into two categories, human and natural causes. It is now a global concern that the climatic changes occurring today have been speeded up because of man's activities. The natural variability and the climate fluctuations of the climat e system have always been part of the Earth’s history however there have been changes in concentrations of greenhouse gases in the atmosphere growing at an unprecedented rate and magnitude. The United Nations, governments and many top scientists around the world believe that we must act now to stabilise and arrest further changes. To understand climate change fully, the causes of climate change must be first identified. Scientists divide the causes into two categories, natural and human causes. Back to top\nWhat Causes Climate Change in detail\nNatural Causes of Climate Change\nThe earth’s climate is influenced and changed through natural causes like volcanic eruptions, ocean current, the earth’s orbital changes and solar variations. Volcanic eruptions - When a volcano erupts it throws out large volumes of sulphur dioxide (SO2), water vapour, dust, and ash into the atmosphere. Large volumes of gases and ash can influence climatic patterns for years by increasing planetary reflectivity causing atmospheric cooling.",
        "start_char": "0",
        "end_char": "3616"
      }
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#0_1655118562",
      "score": 9.7756,
      "rank": 2,
      "doc": {
        "url": "https://whatsyourimpact.org/climate-change/causes",
        "title": "What causes climate change? | What's Your Impact",
        "headings": "What causes climate change?\nWhat causes climate change?\nWhat causes climate change: Forcing\nHuman sources\nNatural sources\nWhat causes climate change: Feedback\n",
        "segment": "What causes climate change? | What's Your Impact\nYou are here\nHome\n»\nClimate change\n»\nClimate change causes\nWhat causes climate change? Share Tweet\nWhat causes climate change? Climate changes are caused by changes in the total amount of energy from the Sun that is kept in the Earth's atmosphere. 1 This change in energy is then distributed around the globe by winds, ocean currents, and other mechanisms to affect the climates of different regions. Both natural processes and human activities can change the rate of energy absorbed by the atmosphere. 1 Factors that can affect climate change can be divided between feedback and forcing processes. Forcing processes are external factors that can cause changes directly to our climate, where as feedback processes amplify or diminish the effect of climate forcing. 2\nWhat causes climate change: Forcing\nA forcing process changes the proportion of incoming solar radiation and outgoing infrared radiation that is kept in the Earth's atmosphere and so changes the amount of energy within global weather patterns.",
        "start_char": "0",
        "end_char": "1059"
      }
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#1_1655120049",
      "score": 9.6667,
      "rank": 3,
      "doc": {
        "url": "https://whatsyourimpact.org/climate-change/causes",
        "title": "What causes climate change? | What's Your Impact",
        "headings": "What causes climate change?\nWhat causes climate change?\nWhat causes climate change: Forcing\nHuman sources\nNatural sources\nWhat causes climate change: Feedback\n",
        "segment": "Both natural processes and human activities can change the rate of energy absorbed by the atmosphere. 1 Factors that can affect climate change can be divided between feedback and forcing processes. Forcing processes are external factors that can cause changes directly to our climate, where as feedback processes amplify or diminish the effect of climate forcing. 2\nWhat causes climate change: Forcing\nA forcing process changes the proportion of incoming solar radiation and outgoing infrared radiation that is kept in the Earth's atmosphere and so changes the amount of energy within global weather patterns. Forcings are external processes that affect the climate system. Examples of forcings include volcanic eruptions and increased greenhouse gas concentrations. 3 4\nThe Earth's climate has never been completely static and in the past the planet's climate has changed due to natural causes but the climate changes seen today are being caused by the increase of carbon dioxide and other greenhouse gases in the atmosphere by humans. 5\nLet's take a look at the different types of forcings that can affect climate change: Human sources\nChanges in greenhouse gas concentrations\nDeforestation and land use changes\nSulfate aerosols\nSoot particles (black carbon)\nNatural sources\nVariations in the Earth's orbital characteristics\nVolcanic eruptions\nVariations in solar output\nNatural aerosols\nWhat causes climate change:",
        "start_char": "449",
        "end_char": "1867"
      }
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#4_1655125064",
      "score": 9.6592,
      "rank": 4,
      "doc": {
        "url": "https://whatsyourimpact.org/climate-change/causes",
        "title": "What causes climate change? | What's Your Impact",
        "headings": "What causes climate change?\nWhat causes climate change?\nWhat causes climate change: Forcing\nHuman sources\nNatural sources\nWhat causes climate change: Feedback\n",
        "segment": "Let's take a look at the different types of feedbacks that can affect climate change: Natural sources\nWater vapor\nIce cover reflectivity\nClouds\nMore info: Climate Change: Causes - NASA\nCauses of Climate Change - US EPA\nWhat causes the Earth's climate to change? - British Geological Survey\nAre human activities causing climate change? - Australian Academy of Science\n1. a. b. The Royal Society. Climate change: A Summary of the Science. London:",
        "start_char": "2460",
        "end_char": "2905"
      }
    },
    {
      "docid": "msmarco_v2.1_doc_00_1066948891#5_1934385985",
      "score": 9.6442,
      "rank": 5,
      "doc": {
        "url": "http://climatechange.gc.ca/default.asp?lang=En&n=65CD73F4-1",
        "title": "Causes of climate change - Canada.ca",
        "headings": "Causes of climate change\nCauses of climate change\nWhat is the most important cause of climate change?\nWhat are climate forcers?\nWhat can be done about climate change?\nWhat is climate change?\nDoes climate change have natural causes?\nHuman causes\nShort-lived and long-lived climate forcers\n",
        "segment": "This indicates how difficult the challenge is. What is climate change? Climate change is a long-term shift in weather conditions identified by changes in temperature, precipitation, winds, and other indicators. Climate change can involve both changes in average conditions and changes in variability, including, for example, extreme events. The earth's climate is naturally variable on all time scales. However, its long-term state and average temperature are regulated by the balance between incoming and outgoing energy, which determines the Earth's energy balance. Any factor that causes a sustained change to the amount of incoming energy or the amount of outgoing energy can lead to climate change. Different factors operate on different time scales, and not all of those factors that have been responsible for changes in earth's climate in the distant past are relevant to contemporary climate change. Factors that cause climate change can be divided into two categories ­- those related to natural processes and those related to human activity. In addition to natural causes of climate change, changes internal to the climate system, such as variations.",
        "start_char": "2321",
        "end_char": "3481"
      }
    }
  ]
}

% curl "http://localhost:8081/v1/msmarco-v2.1-doc-segmented/search?query=what%20causes%20climate%20change&hits=5&parse=false" | jq
{
  "api": "v1",
  "index": "msmarco-v2.1-doc-segmented",
  "query": {
    "text": "what causes climate change"
  },
  "candidates": [
    {
      "docid": "msmarco_v2.1_doc_05_273042196#0_275558320",
      "score": 9.8328,
      "rank": 1,
      "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_05_273042196#0_275558320\",\n  \"url\" : \"http://www.climatechangechallenge.org/Resource%20Centre/Climate-Change/3-what_causes_climate_change.htm\",\n  \"title\" : \"Causes of Climate Change\",\n  \"headings\" : \"Page Contents - Causes of Climate Change\\nPage Contents - Causes of Climate Change\",\n  \"segment\" : \"Causes of Climate Change\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nthe\\nimpact and effects of climate change\\nwhat is\\nclimate change\\nwhat are greenhouse gases\\ncauses\\nof climate change\\nwhat is\\ncarbon footprint\\nwhat\\naction can we take\\nfeatured articles\\nHow recycling mobile phones can benefit the environment\\nGlobal warming, fact or fable\\nHow energy companies rip you off\\nClimate change: a summary of the science by The Royal Society\\nBellamy, Climate Change not Man Made\\nCombating climate change- China’s contribution to the expansion of Africa’s renewable energy sector\\nThe Climate Change Challenge for British Woodland\\nDo Volcanoes cause climate change\\nDisposable Nappies (diapers) - No Worse for the Environment Than Cloth Nappies\\nWalking to the shops damages planet more than going by car\\nCauses of Climate Change slammed by 140 Scientists\\n86 million Americans without healthcare\\nThe truth about climate change\\nRenewable energy from biomass and biofuel\\nThe 10 big energy myths\\nThe electric car\\nLessons from the Copenhagen Climate Change Conference\\nPage Contents - Causes of Climate Change\\nCauses of Climate Change\\nWhat Causes Climate Change in detail\\nMain Contributors and Causes of Climate Change\\nAgriculture as a Contributor to the Causes of Climate Change\\nDeforestation as a contributor to the Causes of Climate change\\nIncrease in global temperatures\\nShattering the Green myths\\nCauses and Effects of Climate Change in Ethiopia\\nExisting Work and Knowledge regarding climate change\\nThe Royal Society's 2010 Report looks at the science behind the causes of Climate Change\\nNitrous Oxide No Laughing Matter\\nScientific Basis of Climate Change and Causes\\nClimate Change and the IPCC (Intergovernmental Panel on Climate Change)\\nNASA's Earth Observation System monitoring the changing climate\\nLivestock and Climate Change Causes\\nEmotional Causes of Climate Change Denial\\nUnited States Global Research Climate Program - \\\"human activities have increased additional heat - trapping gases\\\"\\nCauses of climate change millions of years ago\\nBlack carbon now second largest cause of climate change and key human contributor to global warming\\nGreedy Lying Bastards - a powerful film about the people who cast doubts on the causes of climate change\\nJoint science academies’ statement on the Global response to climate change and causes\\nCauses of Climate Change\\nThe causes of climate change can be divided into two categories, human and natural causes. It is now a global concern that the climatic changes occurring today have been speeded up because of man's activities. The natural variability and the climate fluctuations of the climat e system have always been part of the Earth’s history however there have been changes in concentrations of greenhouse gases in the atmosphere growing at an unprecedented rate and magnitude. The United Nations, governments and many top scientists around the world believe that we must act now to stabilise and arrest further changes. To understand climate change fully, the causes of climate change must be first identified. Scientists divide the causes into two categories, natural and human causes. Back to top\\nWhat Causes Climate Change in detail\\nNatural Causes of Climate Change\\nThe earth’s climate is influenced and changed through natural causes like volcanic eruptions, ocean current, the earth’s orbital changes and solar variations. Volcanic eruptions - When a volcano erupts it throws out large volumes of sulphur dioxide (SO2), water vapour, dust, and ash into the atmosphere. Large volumes of gases and ash can influence climatic patterns for years by increasing planetary reflectivity causing atmospheric cooling.\",\n  \"start_char\" : 0,\n  \"end_char\" : 3616\n}"
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#0_1655118562",
      "score": 9.7756,
      "rank": 2,
      "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_31_828026264#0_1655118562\",\n  \"url\" : \"https://whatsyourimpact.org/climate-change/causes\",\n  \"title\" : \"What causes climate change? | What's Your Impact\",\n  \"headings\" : \"What causes climate change?\\nWhat causes climate change?\\nWhat causes climate change: Forcing\\nHuman sources\\nNatural sources\\nWhat causes climate change: Feedback\\n\",\n  \"segment\" : \"What causes climate change? | What's Your Impact\\nYou are here\\nHome\\n»\\nClimate change\\n»\\nClimate change causes\\nWhat causes climate change? Share Tweet\\nWhat causes climate change? Climate changes are caused by changes in the total amount of energy from the Sun that is kept in the Earth's atmosphere. 1 This change in energy is then distributed around the globe by winds, ocean currents, and other mechanisms to affect the climates of different regions. Both natural processes and human activities can change the rate of energy absorbed by the atmosphere. 1 Factors that can affect climate change can be divided between feedback and forcing processes. Forcing processes are external factors that can cause changes directly to our climate, where as feedback processes amplify or diminish the effect of climate forcing. 2\\nWhat causes climate change: Forcing\\nA forcing process changes the proportion of incoming solar radiation and outgoing infrared radiation that is kept in the Earth's atmosphere and so changes the amount of energy within global weather patterns.\",\n  \"start_char\" : 0,\n  \"end_char\" : 1059\n}"
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#1_1655120049",
      "score": 9.6667,
      "rank": 3,
      "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_31_828026264#1_1655120049\",\n  \"url\" : \"https://whatsyourimpact.org/climate-change/causes\",\n  \"title\" : \"What causes climate change? | What's Your Impact\",\n  \"headings\" : \"What causes climate change?\\nWhat causes climate change?\\nWhat causes climate change: Forcing\\nHuman sources\\nNatural sources\\nWhat causes climate change: Feedback\\n\",\n  \"segment\" : \"Both natural processes and human activities can change the rate of energy absorbed by the atmosphere. 1 Factors that can affect climate change can be divided between feedback and forcing processes. Forcing processes are external factors that can cause changes directly to our climate, where as feedback processes amplify or diminish the effect of climate forcing. 2\\nWhat causes climate change: Forcing\\nA forcing process changes the proportion of incoming solar radiation and outgoing infrared radiation that is kept in the Earth's atmosphere and so changes the amount of energy within global weather patterns. Forcings are external processes that affect the climate system. Examples of forcings include volcanic eruptions and increased greenhouse gas concentrations. 3 4\\nThe Earth's climate has never been completely static and in the past the planet's climate has changed due to natural causes but the climate changes seen today are being caused by the increase of carbon dioxide and other greenhouse gases in the atmosphere by humans. 5\\nLet's take a look at the different types of forcings that can affect climate change: Human sources\\nChanges in greenhouse gas concentrations\\nDeforestation and land use changes\\nSulfate aerosols\\nSoot particles (black carbon)\\nNatural sources\\nVariations in the Earth's orbital characteristics\\nVolcanic eruptions\\nVariations in solar output\\nNatural aerosols\\nWhat causes climate change:\",\n  \"start_char\" : 449,\n  \"end_char\" : 1867\n}"
    },
    {
      "docid": "msmarco_v2.1_doc_31_828026264#4_1655125064",
      "score": 9.6592,
      "rank": 4,
      "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_31_828026264#4_1655125064\",\n  \"url\" : \"https://whatsyourimpact.org/climate-change/causes\",\n  \"title\" : \"What causes climate change? | What's Your Impact\",\n  \"headings\" : \"What causes climate change?\\nWhat causes climate change?\\nWhat causes climate change: Forcing\\nHuman sources\\nNatural sources\\nWhat causes climate change: Feedback\\n\",\n  \"segment\" : \"Let's take a look at the different types of feedbacks that can affect climate change: Natural sources\\nWater vapor\\nIce cover reflectivity\\nClouds\\nMore info: Climate Change: Causes - NASA\\nCauses of Climate Change - US EPA\\nWhat causes the Earth's climate to change? - British Geological Survey\\nAre human activities causing climate change? - Australian Academy of Science\\n1. a. b. The Royal Society. Climate change: A Summary of the Science. London:\",\n  \"start_char\" : 2460,\n  \"end_char\" : 2905\n}"
    },
    {
      "docid": "msmarco_v2.1_doc_00_1066948891#5_1934385985",
      "score": 9.6442,
      "rank": 5,
      "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_00_1066948891#5_1934385985\",\n  \"url\" : \"http://climatechange.gc.ca/default.asp?lang=En&n=65CD73F4-1\",\n  \"title\" : \"Causes of climate change - Canada.ca\",\n  \"headings\" : \"Causes of climate change\\nCauses of climate change\\nWhat is the most important cause of climate change?\\nWhat are climate forcers?\\nWhat can be done about climate change?\\nWhat is climate change?\\nDoes climate change have natural causes?\\nHuman causes\\nShort-lived and long-lived climate forcers\\n\",\n  \"segment\" : \"This indicates how difficult the challenge is. What is climate change? Climate change is a long-term shift in weather conditions identified by changes in temperature, precipitation, winds, and other indicators. Climate change can involve both changes in average conditions and changes in variability, including, for example, extreme events. The earth's climate is naturally variable on all time scales. However, its long-term state and average temperature are regulated by the balance between incoming and outgoing energy, which determines the Earth's energy balance. Any factor that causes a sustained change to the amount of incoming energy or the amount of outgoing energy can lead to climate change. Different factors operate on different time scales, and not all of those factors that have been responsible for changes in earth's climate in the distant past are relevant to contemporary climate change. Factors that cause climate change can be divided into two categories ­- those related to natural processes and those related to human activity. In addition to natural causes of climate change, changes internal to the climate system, such as variations.\",\n  \"start_char\" : 2321,\n  \"end_char\" : 3481\n}"
    }
  ]
}


% curl "http://localhost:8081/v1/msmarco-v2.1-doc-segmented/doc/msmarco_v2.1_doc_00_1066948891%235_1934385985" | jq
{
  "api": "v1",
  "index": "msmarco-v2.1-doc-segmented",
  "docid": "msmarco_v2.1_doc_00_1066948891#5_1934385985",
  "doc": {
    "url": "http://climatechange.gc.ca/default.asp?lang=En&n=65CD73F4-1",
    "title": "Causes of climate change - Canada.ca",
    "headings": "Causes of climate change\nCauses of climate change\nWhat is the most important cause of climate change?\nWhat are climate forcers?\nWhat can be done about climate change?\nWhat is climate change?\nDoes climate change have natural causes?\nHuman causes\nShort-lived and long-lived climate forcers\n",
    "segment": "This indicates how difficult the challenge is. What is climate change? Climate change is a long-term shift in weather conditions identified by changes in temperature, precipitation, winds, and other indicators. Climate change can involve both changes in average conditions and changes in variability, including, for example, extreme events. The earth's climate is naturally variable on all time scales. However, its long-term state and average temperature are regulated by the balance between incoming and outgoing energy, which determines the Earth's energy balance. Any factor that causes a sustained change to the amount of incoming energy or the amount of outgoing energy can lead to climate change. Different factors operate on different time scales, and not all of those factors that have been responsible for changes in earth's climate in the distant past are relevant to contemporary climate change. Factors that cause climate change can be divided into two categories ­- those related to natural processes and those related to human activity. In addition to natural causes of climate change, changes internal to the climate system, such as variations.",
    "start_char": "2321",
    "end_char": "3481"
  }
}

% curl "http://localhost:8081/v1/msmarco-v2.1-doc-segmented/doc/msmarco_v2.1_doc_00_1066948891%235_1934385985?parse=false" | jq
{
  "api": "v1",
  "index": "msmarco-v2.1-doc-segmented",
  "docid": "msmarco_v2.1_doc_00_1066948891#5_1934385985",
  "doc": "{\n  \"docid\" : \"msmarco_v2.1_doc_00_1066948891#5_1934385985\",\n  \"url\" : \"http://climatechange.gc.ca/default.asp?lang=En&n=65CD73F4-1\",\n  \"title\" : \"Causes of climate change - Canada.ca\",\n  \"headings\" : \"Causes of climate change\\nCauses of climate change\\nWhat is the most important cause of climate change?\\nWhat are climate forcers?\\nWhat can be done about climate change?\\nWhat is climate change?\\nDoes climate change have natural causes?\\nHuman causes\\nShort-lived and long-lived climate forcers\\n\",\n  \"segment\" : \"This indicates how difficult the challenge is. What is climate change? Climate change is a long-term shift in weather conditions identified by changes in temperature, precipitation, winds, and other indicators. Climate change can involve both changes in average conditions and changes in variability, including, for example, extreme events. The earth's climate is naturally variable on all time scales. However, its long-term state and average temperature are regulated by the balance between incoming and outgoing energy, which determines the Earth's energy balance. Any factor that causes a sustained change to the amount of incoming energy or the amount of outgoing energy can lead to climate change. Different factors operate on different time scales, and not all of those factors that have been responsible for changes in earth's climate in the distant past are relevant to contemporary climate change. Factors that cause climate change can be divided into two categories ­- those related to natural processes and those related to human activity. In addition to natural causes of climate change, changes internal to the climate system, such as variations.\",\n  \"start_char\" : 2321,\n  \"end_char\" : 3481\n}"
}

@lintool
Copy link
Copy Markdown
Member Author

lintool commented Mar 24, 2026

% curl "http://localhost:8081/v1/beir-v1.0.0-nfcorpus.flat/search?query=how%20to%20treat%20high%20blood%20pressure&hits=5" | jq
{
  "api": "v1",
  "index": "beir-v1.0.0-nfcorpus.flat",
  "query": {
    "text": "how to treat high blood pressure"
  },
  "candidates": [
    {
      "docid": "MED-5290",
      "score": 8.2143,
      "rank": 1,
      "doc": {
        "title": "By how much does dietary salt reduction lower blood pressure? III--Analysis of data from trials of salt reduction.",
        "text": "OBJECTIVE: To determine whether the reduction in blood pressure achieved in trials of dietary salt reduction is quantitatively consistent with estimates derived from blood pressure and sodium intake in different populations, and, if so, to estimate the impact of reducing dietary salt on mortality from stroke and ischaemic heart disease. DESIGN: Analysis of the results of 68 crossover trials and 10 randomised controlled trials of dietary salt reduction. MAIN OUTCOME MEASURE: Comparison of observed reductions in systolic blood pressure for each trial with predicted values calculated from between population analysis. RESULTS: In the 45 trials in which salt reduction lasted four weeks or less the observed reductions in blood pressure were less than those predicted, with the difference between observed and predicted reductions being greatest in the trials of shortest duration. In the 33 trials lasting five weeks or longer the predicted reductions in individual trials closely matched a wide range of observed reductions. This applied for all age groups and for people with both high and normal levels of blood pressure. In people aged 50-59 years a reduction in daily sodium intake of 50 mmol (about 3 g of salt), attainable by moderate dietary salt reduction would, after a few weeks, lower systolic blood pressure by an average of 5 mm Hg, and by 7 mm Hg in those with high blood pressure (170 mm Hg); diastolic blood pressure would be lowered by about half as much. It is estimated that such a reduction in salt intake by a whole Western population would reduce the incidence of stroke by 22% and of ischaemic heart disease by 16% [corrected]. CONCLUSIONS: The results from the trials support the estimates from the observational data in the accompanying two papers. The effect of universal moderate dietary salt reduction on mortality from stroke and ischaemic heart disease would be substantial--larger, indeed, than could be achieved by fully implementing recommended policy for treating high blood pressure with drugs. However, reduction also in the amount of salt added to processed foods would lower blood pressure by at least twice as much and prevent some 75,000 [corrected] deaths a year in Britain as well as much disability.",
        "metadata": {
          "url": "http://www.ncbi.nlm.nih.gov/pubmed/1827353"
        }
      }
    },
    {
      "docid": "MED-5299",
      "score": 6.3503,
      "rank": 2,
      "doc": {
        "title": "The Preventable Causes of Death in the United States: Comparative Risk Assessment of Dietary, Lifestyle, and Metabolic Risk Factors",
        "text": "Background Knowledge of the number of deaths caused by risk factors is needed for health policy and priority setting. Our aim was to estimate the mortality effects of the following 12 modifiable dietary, lifestyle, and metabolic risk factors in the United States (US) using consistent and comparable methods: high blood glucose, low-density lipoprotein (LDL) cholesterol, and blood pressure; overweight–obesity; high dietary trans fatty acids and salt; low dietary polyunsaturated fatty acids, omega-3 fatty acids (seafood), and fruits and vegetables; physical inactivity; alcohol use; and tobacco smoking. Methods and Findings We used data on risk factor exposures in the US population from nationally representative health surveys and disease-specific mortality statistics from the National Center for Health Statistics. We obtained the etiological effects of risk factors on disease-specific mortality, by age, from systematic reviews and meta-analyses of epidemiological studies that had adjusted (i) for major potential confounders, and (ii) where possible for regression dilution bias. We estimated the number of disease-specific deaths attributable to all non-optimal levels of each risk factor exposure, by age and sex. In 2005, tobacco smoking and high blood pressure were responsible for an estimated 467,000 (95% confidence interval [CI] 436,000–500,000) and 395,000 (372,000–414,000) deaths, accounting for about one in five or six deaths in US adults. Overweight–obesity (216,000; 188,000–237,000) and physical inactivity (191,000; 164,000–222,000) were each responsible for nearly 1 in 10 deaths. High dietary salt (102,000; 97,000–107,000), low dietary omega-3 fatty acids (84,000; 72,000–96,000), and high dietary trans fatty acids (82,000; 63,000–97,000) were the dietary risks with the largest mortality effects. Although 26,000 (23,000–40,000) deaths from ischemic heart disease, ischemic stroke, and diabetes were averted by current alcohol use, they were outweighed by 90,000 (88,000–94,000) deaths from other cardiovascular diseases, cancers, liver cirrhosis, pancreatitis, alcohol use disorders, road traffic and other injuries, and violence. Conclusions Smoking and high blood pressure, which both have effective interventions, are responsible for the largest number of deaths in the US. Other dietary, lifestyle, and metabolic risk factors for chronic diseases also cause a substantial number of deaths in the US. Please see later in the article for Editors' Summary Editors' Summary A number of modifiable factors are responsible for many premature or preventable deaths. For example, being overweight or obese shortens life expectancy, while half of all long-term tobacco smokers in Western populations will die prematurely from a disease directly related to smoking. Modifiable risk factors fall into three main groups. First, there are lifestyle risk factors. These include tobacco smoking, physical inactivity, and excessive alcohol use (small amounts of alcohol may actually prevent diabetes and some types of heart disease and stroke). Second, there are dietary risk factors such as a high salt intake and a low intake of fruits and vegetables. Finally, there are “metabolic risk factors,” which shorten life expectancy by increasing a person's chances of developing cardiovascular disease (in particular, heart problems and strokes) and diabetes. Metabolic risk factors include having high blood pressure or blood cholesterol and being overweight or obese. Why Was This Study Done? It should be possible to reduce preventable deaths by changing modifiable risk factors through introducing public health policies, programs and regulations that reduce exposures to these risk factors. However, it is important to know how many deaths are caused by each risk factor before developing policies and programs that aim to improve a nation's health. Although previous studies have provided some information on the numbers of premature deaths caused by modifiable risk factors, there are two problems with these studies. First, they have not used consistent and comparable methods to estimate the number of deaths attributable to different risk factors. Second, they have rarely considered the effects of dietary and metabolic risk factors. In this new study, the researchers estimate the number of deaths due to 12 different modifiable dietary, lifestyle, and metabolic risk factors for the United States population. They use a method called “comparative risk assessment.” This approach estimates the number of deaths that would be prevented if current distributions of risk factor exposures were changed to hypothetical optimal distributions. What Did the Researchers Do and Find? The researchers extracted data on exposures to these 12 selected risk factors from US national health surveys, and they obtained information on deaths from difference diseases for 2005 from the US National Center for Health Statistics. They used previously published studies to estimate how much each risk factor increases the risk of death from each disease. The researchers then used a mathematical formula to estimate the numbers of deaths caused by each risk factor. Of the 2.5 million US deaths in 2005, they estimate that nearly half a million were associated with tobacco smoking and about 400,000 were associated with high blood pressure. These two risk factors therefore each accounted for about 1 in 5 deaths in US adults. Overweight–obesity and physical inactivity were each responsible for nearly 1 in 10 deaths. Among the dietary factors examined, high dietary salt intake had the largest effect, being responsible for 4% of deaths in adults. Finally, while alcohol use prevented 26,000 deaths from ischemic heart disease, ischemic stroke, and diabetes, the researchers estimate that it caused 90,000 deaths from other types of cardiovascular diseases, other medical conditions, and road traffic accidents and violence. What Do These Findings Mean? These findings indicate that smoking and high blood pressure are responsible for the largest number of preventable deaths in the US, but that several other modifiable risk factors also cause many deaths. Although the accuracy of some of the estimates obtained in this study will be affected by the quality of the data used, these findings suggest that targeting a handful of risk factors could greatly reduce premature mortality in the US. The findings might also apply to other countries, although the risk factors responsible for most preventable deaths may vary between countries. Importantly, effective individual-level and population-wide interventions are already available to reduce people's exposure to the two risk factors responsible for most preventable deaths in the US. The researchers also suggest that combinations of regulation, pricing, and education have the potential to reduce the exposure of US residents to other risk factors that are likely to shorten their lives. Additional Information Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000058.",
        "metadata": {
          "url": "http://www.ncbi.nlm.nih.gov/pubmed/19399161"
        }
      }
    },
    {
      "docid": "MED-4417",
      "score": 5.7738,
      "rank": 3,
      "doc": {
        "title": "Glycated haemoglobin and blood pressure-lowering effect of cinnamon in multi-ethnic Type 2 diabetic patients in the UK: a randomized, placebo-contr...",
        "text": "AIMS: To determine the blood glucose lowering effect of cinnamon on HbA1c, blood pressure and lipid profiles in people with type 2 diabetes. METHODS: 58 type 2 diabetic patients (25 males and 33 females), aged 54.9 ± 9.8, treated only with hypoglycemic agents and with an HbA1c more than 7% were randomly assigned to receive either 2g of cinnamon or placebo daily for 12 weeks. RESULTS: After intervention, the mean HbA1c was significantly decreased (P<0.005) in the cinnamon group (8.22% to 7.86%) compared with placebo group (8.55% to 8.68%). Mean systolic and diastolic blood pressures (SBP and DBP) were also significantly reduced (P<0.001) after 12 weeks in the cinnamon group (SBP: 132.6 to 129.2 mmHg and DBP: 85.2 to 80.2 mmHg) compared with the placebo group (SBP: 134.5 to 134.9 mmHg and DBP: 86.8 to 86.1 mmHg). A significant reduction in fasting plasma glucose (FPG), waist circumference and body mass index (BMI) was observed at week 12 compared to baseline in the cinnamon group, however, the changes were not significant when compared to placebo group. There were no significant differences in serum lipid profiles of total cholesterol, triglycerides, HDL and LDL cholesterols neither between nor within the groups. CONCLUSIONS: Intake of 2g of cinnamon for 12 weeks significantly reduces the HbA1c, SBP and DBP among poorly controlled type 2 diabetes patients. Cinnamon supplementation could be considered as an additional dietary supplement option to regulate blood glucose and blood pressure levels along with conventional medications to treat type 2 diabetes mellitus. © 2010 The Authors. Diabetic Medicine © 2010 Diabetes UK.",
        "metadata": {
          "url": "http://www.ncbi.nlm.nih.gov/pubmed/20854384"
        }
      }
    },
    {
      "docid": "MED-5331",
      "score": 5.7198,
      "rank": 4,
      "doc": {
        "title": "Influencing public nutrition for non-communicable disease prevention: from community intervention to national programme--experiences from Finland.",
        "text": "A global health transition is currently underway. The burden of non-communicable diseases (NCDs) is increasing rapidly in the developing world, very much as a result of changes in lifestyles. In addition to changes in tobacco use and physical activity, major changes are taking place in diets, contributing greatly to the growing epidemic of NCD. Thus, a huge global public health challenge is how to influence the trends in diet and nutrition for effective global NCD prevention. The health transition took place rapidly in Finland after World War II and mortality from cardiovascular disease (CVD) was exceptionally high. The North Karelia Project was launched in 1972 as a community-based, and later as a national, programme to influence diet and other lifestyles that are crucial in the prevention of CVD. The intervention had a strong theory base and it employed comprehensive strategies. Broad community organisation and the strong participation of people were the key elements. Evaluation has shown how the diet (particularly fat consumption) has changed and how these changes have led to a major reduction in population serum cholesterol and blood pressure levels. It has also shown how ischaemic heart disease mortality in a working-age population has declined by 73% in North Karelia and by 65% in the whole country from 1971 to 1995. Although Finland is an industrialised country, North Karelia was rural, of rather low socio-economic level and with many social problems in the 1970s and 1980s. The project was based on low-cost intervention activities, where people's participation and community organisations played a key role. Comprehensive interventions in the community were eventually supported by national activities--from expert guidelines and media activities to industry collaboration and policy. Similar principles for nutrition intervention programmes could be used in developing countries, obviously tailored to the local conditions. This paper discusses the experiences of the North Karelia Project in the light of needs from the less-industrialised countries and makes some general recommendations.",
        "metadata": {
          "url": "http://www.ncbi.nlm.nih.gov/pubmed/12027291"
        }
      }
    },
    {
      "docid": "MED-1258",
      "score": 5.685,
      "rank": 5,
      "doc": {
        "title": "The effect of combining plant sterols, soy protein, viscous fibers, and almonds in treating hypercholesterolemia.",
        "text": "Reductions in low-density lipoprotein-cholesterol (LDL-C) result from diets containing almonds, or diets that are either low in saturated fat or high in viscous fibers, soy proteins, or plant sterols. We have therefore combined all of these interventions in a single diet (portfolio diet) to determine whether cholesterol reductions could be achieved of similar magnitude to those reported in recent statin trials which reduced cardiovascular events. Twenty-five hyperlipidemic subjects consumed either a portfolio diet (n=13), very low in saturated fat and high in plant sterols (1.2 g/1,000 kcal), soy protein (16.2 g/1,000 kcal), viscous fibers (8.3 g/1,000 kcal), and almonds (16.6 g/1,000 kcal), or a low-saturated fat diet (n=12) based on whole-wheat cereals and low-fat dairy foods. Fasting blood, blood pressure, and body weight were obtained at weeks 0, 2, and 4 of each phase. LDL-C was reduced by 12.1% +/- 2.4% (P<.001) on the low-fat diet and by 35.0% +/- 3.1% (P<.001) on the portfolio diet, which also reduced the ratio of LDL-C to high-density lipoprotein-cholesterol (HDL-C) significantly (30.0% +/- 3.5%; P<.001). The reductions in LDL-C and the LDL:HDL-C ratio were both significantly lower on the portfolio diet than on the control diet (P<.001 and P<.001, respectively). Mean weight loss was similar on test and control diets (1.0 kg and 0.9 kg, respectively). No difference was seen in blood pressure, HDL-C, serum triglycerides, lipoprotein(a) [Lp(a)], or homocysteine concentrations between diets. Combining a number of foods and food components in a single dietary portfolio may lower LDL-C similarly to statins and so increase the potential effectiveness of dietary therapy.",
        "metadata": {
          "url": "http://www.ncbi.nlm.nih.gov/pubmed/14624410"
        }
      }
    }
  ]
}

% curl "http://localhost:8081/v1/beir-v1.0.0-nfcorpus.flat/search?query=how%20to%20treat%20high%20blood%20pressure&hits=5&parse=false" | jq
{
  "api": "v1",
  "index": "beir-v1.0.0-nfcorpus.flat",
  "query": {
    "text": "how to treat high blood pressure"
  },
  "candidates": [
    {
      "docid": "MED-5290",
      "score": 8.2143,
      "rank": 1,
      "doc": "{\n  \"_id\" : \"MED-5290\",\n  \"title\" : \"By how much does dietary salt reduction lower blood pressure? III--Analysis of data from trials of salt reduction.\",\n  \"text\" : \"OBJECTIVE: To determine whether the reduction in blood pressure achieved in trials of dietary salt reduction is quantitatively consistent with estimates derived from blood pressure and sodium intake in different populations, and, if so, to estimate the impact of reducing dietary salt on mortality from stroke and ischaemic heart disease. DESIGN: Analysis of the results of 68 crossover trials and 10 randomised controlled trials of dietary salt reduction. MAIN OUTCOME MEASURE: Comparison of observed reductions in systolic blood pressure for each trial with predicted values calculated from between population analysis. RESULTS: In the 45 trials in which salt reduction lasted four weeks or less the observed reductions in blood pressure were less than those predicted, with the difference between observed and predicted reductions being greatest in the trials of shortest duration. In the 33 trials lasting five weeks or longer the predicted reductions in individual trials closely matched a wide range of observed reductions. This applied for all age groups and for people with both high and normal levels of blood pressure. In people aged 50-59 years a reduction in daily sodium intake of 50 mmol (about 3 g of salt), attainable by moderate dietary salt reduction would, after a few weeks, lower systolic blood pressure by an average of 5 mm Hg, and by 7 mm Hg in those with high blood pressure (170 mm Hg); diastolic blood pressure would be lowered by about half as much. It is estimated that such a reduction in salt intake by a whole Western population would reduce the incidence of stroke by 22% and of ischaemic heart disease by 16% [corrected]. CONCLUSIONS: The results from the trials support the estimates from the observational data in the accompanying two papers. The effect of universal moderate dietary salt reduction on mortality from stroke and ischaemic heart disease would be substantial--larger, indeed, than could be achieved by fully implementing recommended policy for treating high blood pressure with drugs. However, reduction also in the amount of salt added to processed foods would lower blood pressure by at least twice as much and prevent some 75,000 [corrected] deaths a year in Britain as well as much disability.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/1827353\"\n  }\n}"
    },
    {
      "docid": "MED-5299",
      "score": 6.3503,
      "rank": 2,
      "doc": "{\n  \"_id\" : \"MED-5299\",\n  \"title\" : \"The Preventable Causes of Death in the United States: Comparative Risk Assessment of Dietary, Lifestyle, and Metabolic Risk Factors\",\n  \"text\" : \"Background Knowledge of the number of deaths caused by risk factors is needed for health policy and priority setting. Our aim was to estimate the mortality effects of the following 12 modifiable dietary, lifestyle, and metabolic risk factors in the United States (US) using consistent and comparable methods: high blood glucose, low-density lipoprotein (LDL) cholesterol, and blood pressure; overweight–obesity; high dietary trans fatty acids and salt; low dietary polyunsaturated fatty acids, omega-3 fatty acids (seafood), and fruits and vegetables; physical inactivity; alcohol use; and tobacco smoking. Methods and Findings We used data on risk factor exposures in the US population from nationally representative health surveys and disease-specific mortality statistics from the National Center for Health Statistics. We obtained the etiological effects of risk factors on disease-specific mortality, by age, from systematic reviews and meta-analyses of epidemiological studies that had adjusted (i) for major potential confounders, and (ii) where possible for regression dilution bias. We estimated the number of disease-specific deaths attributable to all non-optimal levels of each risk factor exposure, by age and sex. In 2005, tobacco smoking and high blood pressure were responsible for an estimated 467,000 (95% confidence interval [CI] 436,000–500,000) and 395,000 (372,000–414,000) deaths, accounting for about one in five or six deaths in US adults. Overweight–obesity (216,000; 188,000–237,000) and physical inactivity (191,000; 164,000–222,000) were each responsible for nearly 1 in 10 deaths. High dietary salt (102,000; 97,000–107,000), low dietary omega-3 fatty acids (84,000; 72,000–96,000), and high dietary trans fatty acids (82,000; 63,000–97,000) were the dietary risks with the largest mortality effects. Although 26,000 (23,000–40,000) deaths from ischemic heart disease, ischemic stroke, and diabetes were averted by current alcohol use, they were outweighed by 90,000 (88,000–94,000) deaths from other cardiovascular diseases, cancers, liver cirrhosis, pancreatitis, alcohol use disorders, road traffic and other injuries, and violence. Conclusions Smoking and high blood pressure, which both have effective interventions, are responsible for the largest number of deaths in the US. Other dietary, lifestyle, and metabolic risk factors for chronic diseases also cause a substantial number of deaths in the US. Please see later in the article for Editors' Summary Editors' Summary A number of modifiable factors are responsible for many premature or preventable deaths. For example, being overweight or obese shortens life expectancy, while half of all long-term tobacco smokers in Western populations will die prematurely from a disease directly related to smoking. Modifiable risk factors fall into three main groups. First, there are lifestyle risk factors. These include tobacco smoking, physical inactivity, and excessive alcohol use (small amounts of alcohol may actually prevent diabetes and some types of heart disease and stroke). Second, there are dietary risk factors such as a high salt intake and a low intake of fruits and vegetables. Finally, there are “metabolic risk factors,” which shorten life expectancy by increasing a person's chances of developing cardiovascular disease (in particular, heart problems and strokes) and diabetes. Metabolic risk factors include having high blood pressure or blood cholesterol and being overweight or obese. Why Was This Study Done? It should be possible to reduce preventable deaths by changing modifiable risk factors through introducing public health policies, programs and regulations that reduce exposures to these risk factors. However, it is important to know how many deaths are caused by each risk factor before developing policies and programs that aim to improve a nation's health. Although previous studies have provided some information on the numbers of premature deaths caused by modifiable risk factors, there are two problems with these studies. First, they have not used consistent and comparable methods to estimate the number of deaths attributable to different risk factors. Second, they have rarely considered the effects of dietary and metabolic risk factors. In this new study, the researchers estimate the number of deaths due to 12 different modifiable dietary, lifestyle, and metabolic risk factors for the United States population. They use a method called “comparative risk assessment.” This approach estimates the number of deaths that would be prevented if current distributions of risk factor exposures were changed to hypothetical optimal distributions. What Did the Researchers Do and Find? The researchers extracted data on exposures to these 12 selected risk factors from US national health surveys, and they obtained information on deaths from difference diseases for 2005 from the US National Center for Health Statistics. They used previously published studies to estimate how much each risk factor increases the risk of death from each disease. The researchers then used a mathematical formula to estimate the numbers of deaths caused by each risk factor. Of the 2.5 million US deaths in 2005, they estimate that nearly half a million were associated with tobacco smoking and about 400,000 were associated with high blood pressure. These two risk factors therefore each accounted for about 1 in 5 deaths in US adults. Overweight–obesity and physical inactivity were each responsible for nearly 1 in 10 deaths. Among the dietary factors examined, high dietary salt intake had the largest effect, being responsible for 4% of deaths in adults. Finally, while alcohol use prevented 26,000 deaths from ischemic heart disease, ischemic stroke, and diabetes, the researchers estimate that it caused 90,000 deaths from other types of cardiovascular diseases, other medical conditions, and road traffic accidents and violence. What Do These Findings Mean? These findings indicate that smoking and high blood pressure are responsible for the largest number of preventable deaths in the US, but that several other modifiable risk factors also cause many deaths. Although the accuracy of some of the estimates obtained in this study will be affected by the quality of the data used, these findings suggest that targeting a handful of risk factors could greatly reduce premature mortality in the US. The findings might also apply to other countries, although the risk factors responsible for most preventable deaths may vary between countries. Importantly, effective individual-level and population-wide interventions are already available to reduce people's exposure to the two risk factors responsible for most preventable deaths in the US. The researchers also suggest that combinations of regulation, pricing, and education have the potential to reduce the exposure of US residents to other risk factors that are likely to shorten their lives. Additional Information Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1000058.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/19399161\"\n  }\n}"
    },
    {
      "docid": "MED-4417",
      "score": 5.7738,
      "rank": 3,
      "doc": "{\n  \"_id\" : \"MED-4417\",\n  \"title\" : \"Glycated haemoglobin and blood pressure-lowering effect of cinnamon in multi-ethnic Type 2 diabetic patients in the UK: a randomized, placebo-contr...\",\n  \"text\" : \"AIMS: To determine the blood glucose lowering effect of cinnamon on HbA1c, blood pressure and lipid profiles in people with type 2 diabetes. METHODS: 58 type 2 diabetic patients (25 males and 33 females), aged 54.9 ± 9.8, treated only with hypoglycemic agents and with an HbA1c more than 7% were randomly assigned to receive either 2g of cinnamon or placebo daily for 12 weeks. RESULTS: After intervention, the mean HbA1c was significantly decreased (P<0.005) in the cinnamon group (8.22% to 7.86%) compared with placebo group (8.55% to 8.68%). Mean systolic and diastolic blood pressures (SBP and DBP) were also significantly reduced (P<0.001) after 12 weeks in the cinnamon group (SBP: 132.6 to 129.2 mmHg and DBP: 85.2 to 80.2 mmHg) compared with the placebo group (SBP: 134.5 to 134.9 mmHg and DBP: 86.8 to 86.1 mmHg). A significant reduction in fasting plasma glucose (FPG), waist circumference and body mass index (BMI) was observed at week 12 compared to baseline in the cinnamon group, however, the changes were not significant when compared to placebo group. There were no significant differences in serum lipid profiles of total cholesterol, triglycerides, HDL and LDL cholesterols neither between nor within the groups. CONCLUSIONS: Intake of 2g of cinnamon for 12 weeks significantly reduces the HbA1c, SBP and DBP among poorly controlled type 2 diabetes patients. Cinnamon supplementation could be considered as an additional dietary supplement option to regulate blood glucose and blood pressure levels along with conventional medications to treat type 2 diabetes mellitus. © 2010 The Authors. Diabetic Medicine © 2010 Diabetes UK.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/20854384\"\n  }\n}"
    },
    {
      "docid": "MED-5331",
      "score": 5.7198,
      "rank": 4,
      "doc": "{\n  \"_id\" : \"MED-5331\",\n  \"title\" : \"Influencing public nutrition for non-communicable disease prevention: from community intervention to national programme--experiences from Finland.\",\n  \"text\" : \"A global health transition is currently underway. The burden of non-communicable diseases (NCDs) is increasing rapidly in the developing world, very much as a result of changes in lifestyles. In addition to changes in tobacco use and physical activity, major changes are taking place in diets, contributing greatly to the growing epidemic of NCD. Thus, a huge global public health challenge is how to influence the trends in diet and nutrition for effective global NCD prevention. The health transition took place rapidly in Finland after World War II and mortality from cardiovascular disease (CVD) was exceptionally high. The North Karelia Project was launched in 1972 as a community-based, and later as a national, programme to influence diet and other lifestyles that are crucial in the prevention of CVD. The intervention had a strong theory base and it employed comprehensive strategies. Broad community organisation and the strong participation of people were the key elements. Evaluation has shown how the diet (particularly fat consumption) has changed and how these changes have led to a major reduction in population serum cholesterol and blood pressure levels. It has also shown how ischaemic heart disease mortality in a working-age population has declined by 73% in North Karelia and by 65% in the whole country from 1971 to 1995. Although Finland is an industrialised country, North Karelia was rural, of rather low socio-economic level and with many social problems in the 1970s and 1980s. The project was based on low-cost intervention activities, where people's participation and community organisations played a key role. Comprehensive interventions in the community were eventually supported by national activities--from expert guidelines and media activities to industry collaboration and policy. Similar principles for nutrition intervention programmes could be used in developing countries, obviously tailored to the local conditions. This paper discusses the experiences of the North Karelia Project in the light of needs from the less-industrialised countries and makes some general recommendations.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/12027291\"\n  }\n}"
    },
    {
      "docid": "MED-1258",
      "score": 5.685,
      "rank": 5,
      "doc": "{\n  \"_id\" : \"MED-1258\",\n  \"title\" : \"The effect of combining plant sterols, soy protein, viscous fibers, and almonds in treating hypercholesterolemia.\",\n  \"text\" : \"Reductions in low-density lipoprotein-cholesterol (LDL-C) result from diets containing almonds, or diets that are either low in saturated fat or high in viscous fibers, soy proteins, or plant sterols. We have therefore combined all of these interventions in a single diet (portfolio diet) to determine whether cholesterol reductions could be achieved of similar magnitude to those reported in recent statin trials which reduced cardiovascular events. Twenty-five hyperlipidemic subjects consumed either a portfolio diet (n=13), very low in saturated fat and high in plant sterols (1.2 g/1,000 kcal), soy protein (16.2 g/1,000 kcal), viscous fibers (8.3 g/1,000 kcal), and almonds (16.6 g/1,000 kcal), or a low-saturated fat diet (n=12) based on whole-wheat cereals and low-fat dairy foods. Fasting blood, blood pressure, and body weight were obtained at weeks 0, 2, and 4 of each phase. LDL-C was reduced by 12.1% +/- 2.4% (P<.001) on the low-fat diet and by 35.0% +/- 3.1% (P<.001) on the portfolio diet, which also reduced the ratio of LDL-C to high-density lipoprotein-cholesterol (HDL-C) significantly (30.0% +/- 3.5%; P<.001). The reductions in LDL-C and the LDL:HDL-C ratio were both significantly lower on the portfolio diet than on the control diet (P<.001 and P<.001, respectively). Mean weight loss was similar on test and control diets (1.0 kg and 0.9 kg, respectively). No difference was seen in blood pressure, HDL-C, serum triglycerides, lipoprotein(a) [Lp(a)], or homocysteine concentrations between diets. Combining a number of foods and food components in a single dietary portfolio may lower LDL-C similarly to statins and so increase the potential effectiveness of dietary therapy.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/14624410\"\n  }\n}"
    }
  ]
}

% curl "http://localhost:8081/v1/beir-v1.0.0-nfcorpus.flat/doc/MED-1258" | jq
{
  "api": "v1",
  "index": "beir-v1.0.0-nfcorpus.flat",
  "docid": "MED-1258",
  "doc": {
    "title": "The effect of combining plant sterols, soy protein, viscous fibers, and almonds in treating hypercholesterolemia.",
    "text": "Reductions in low-density lipoprotein-cholesterol (LDL-C) result from diets containing almonds, or diets that are either low in saturated fat or high in viscous fibers, soy proteins, or plant sterols. We have therefore combined all of these interventions in a single diet (portfolio diet) to determine whether cholesterol reductions could be achieved of similar magnitude to those reported in recent statin trials which reduced cardiovascular events. Twenty-five hyperlipidemic subjects consumed either a portfolio diet (n=13), very low in saturated fat and high in plant sterols (1.2 g/1,000 kcal), soy protein (16.2 g/1,000 kcal), viscous fibers (8.3 g/1,000 kcal), and almonds (16.6 g/1,000 kcal), or a low-saturated fat diet (n=12) based on whole-wheat cereals and low-fat dairy foods. Fasting blood, blood pressure, and body weight were obtained at weeks 0, 2, and 4 of each phase. LDL-C was reduced by 12.1% +/- 2.4% (P<.001) on the low-fat diet and by 35.0% +/- 3.1% (P<.001) on the portfolio diet, which also reduced the ratio of LDL-C to high-density lipoprotein-cholesterol (HDL-C) significantly (30.0% +/- 3.5%; P<.001). The reductions in LDL-C and the LDL:HDL-C ratio were both significantly lower on the portfolio diet than on the control diet (P<.001 and P<.001, respectively). Mean weight loss was similar on test and control diets (1.0 kg and 0.9 kg, respectively). No difference was seen in blood pressure, HDL-C, serum triglycerides, lipoprotein(a) [Lp(a)], or homocysteine concentrations between diets. Combining a number of foods and food components in a single dietary portfolio may lower LDL-C similarly to statins and so increase the potential effectiveness of dietary therapy.",
    "metadata": {
      "url": "http://www.ncbi.nlm.nih.gov/pubmed/14624410"
    }
  }
}

% curl "http://localhost:8081/v1/beir-v1.0.0-nfcorpus.flat/doc/MED-1258?parse=false" | jq
{
  "api": "v1",
  "index": "beir-v1.0.0-nfcorpus.flat",
  "docid": "MED-1258",
  "doc": "{\n  \"_id\" : \"MED-1258\",\n  \"title\" : \"The effect of combining plant sterols, soy protein, viscous fibers, and almonds in treating hypercholesterolemia.\",\n  \"text\" : \"Reductions in low-density lipoprotein-cholesterol (LDL-C) result from diets containing almonds, or diets that are either low in saturated fat or high in viscous fibers, soy proteins, or plant sterols. We have therefore combined all of these interventions in a single diet (portfolio diet) to determine whether cholesterol reductions could be achieved of similar magnitude to those reported in recent statin trials which reduced cardiovascular events. Twenty-five hyperlipidemic subjects consumed either a portfolio diet (n=13), very low in saturated fat and high in plant sterols (1.2 g/1,000 kcal), soy protein (16.2 g/1,000 kcal), viscous fibers (8.3 g/1,000 kcal), and almonds (16.6 g/1,000 kcal), or a low-saturated fat diet (n=12) based on whole-wheat cereals and low-fat dairy foods. Fasting blood, blood pressure, and body weight were obtained at weeks 0, 2, and 4 of each phase. LDL-C was reduced by 12.1% +/- 2.4% (P<.001) on the low-fat diet and by 35.0% +/- 3.1% (P<.001) on the portfolio diet, which also reduced the ratio of LDL-C to high-density lipoprotein-cholesterol (HDL-C) significantly (30.0% +/- 3.5%; P<.001). The reductions in LDL-C and the LDL:HDL-C ratio were both significantly lower on the portfolio diet than on the control diet (P<.001 and P<.001, respectively). Mean weight loss was similar on test and control diets (1.0 kg and 0.9 kg, respectively). No difference was seen in blood pressure, HDL-C, serum triglycerides, lipoprotein(a) [Lp(a)], or homocysteine concentrations between diets. Combining a number of foods and food components in a single dietary portfolio may lower LDL-C similarly to statins and so increase the potential effectiveness of dietary therapy.\",\n  \"metadata\" : {\n    \"url\" : \"http://www.ncbi.nlm.nih.gov/pubmed/14624410\"\n  }\n}"
}

@lintool lintool marked this pull request as ready for review March 24, 2026 14:52
@lintool
Copy link
Copy Markdown
Member Author

lintool commented Mar 24, 2026

Added sample indexes, added lots of tests.

@codex review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 53660161c4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@sahel-sh
Copy link
Copy Markdown
Member

+1 to the example comments in the PR and the parse flag

@lintool
Copy link
Copy Markdown
Member Author

lintool commented Mar 24, 2026

Removal of me.tongfei.progressbar.ProgressBar is related to refactoring ExtractQueriesAndDocumentsFromTrecRunTest to extend StdOutStdErrRedirectableLuceneTestCase, which caused thread leak issues. Easier to just remove the external progress bar lib.

@lintool lintool merged commit ada1100 into master Mar 24, 2026
3 checks passed
@lintool lintool deleted the issue-3169 branch March 24, 2026 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants