This Article Full Text Full Text (PDF) All Versions of this Article: M1953v1 14/2/164    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Moskovitch, R. Articles by Shahar, Y. Search for Related Content PubMed PubMed Citation Articles by Moskovitch, R. Articles by Shahar, Y.

A Comparative Evaluation of Full-text, Concept-based, and Context-sensitive Search

^a Medical Informatics Research Center, Department of Information Systems Engineering, Ben Gurion University, Beer Sheva, Israel
^b Stanford University, Palo Alto, CA
^c VA Palo Alto Heath Care Center, Palo Alto, CA
^d E&C Medical Intelligence Inc., New York, NY
^e Medical Corps, Israel Defense Force, Israel.

^_* Correspondence and reprints: Robert Moskovitch, Medical Informatics Research Center, Department of Information Systems Engineering, Ben Gurion University of the Negev, Israel, P.O.B. 653, Beer Sheva 84105, Israel. (Email: robertmo{at}bgu.ac.il).

Received for publication: 08/31/05; accepted for publication: 12/11/06.

Objectives: Study comparatively (1) concept-based search, using documents pre-indexed by a conceptual hierarchy; (2) context-sensitive search, using structured, labeled documents; and (3) traditional full-text search. Hypotheses were: (1) more contexts lead to better retrieval accuracy; and (2) adding concept-based search to the other searches would improve upon their baseline performances.

Design: Use our Vaidurya architecture, for search and retrieval evaluation, of structured documents classified by a conceptual hierarchy, on a clinical guidelines test collection.

Measurements: Precision computed at different levels of recall to assess the contribution of the retrieval methods. Comparisons of precisions done with recall set at 0.5, using t-tests.

Results: Performance increased monotonically with the number of query context elements. Adding context-sensitive elements, mean improvement was 11.1% at recall 0.5. With three contexts, mean query precision was 42% ± 17% (95% confidence interval [CI], 31% to 53%); with two contexts, 32% ± 13% (95% CI, 27% to 38%); and one context, 20% ± 9% (95% CI, 15% to 24%). Adding context-based queries to full-text queries monotonically improved precision beyond the 0.4 level of recall. Mean improvement was 4.5% at recall 0.5. Adding concept-based search to full-text search improved precision to 19.4% at recall 0.5.

Conclusions: The study demonstrated usefulness of concept-based and context-sensitive queries for enhancing the precision of retrieval from a digital library of semi-structured clinical guideline documents. Concept-based searches outperformed free-text queries, especially when baseline precision was low. In general, the more ontological elements used in the query, the greater the resulting precision.