{% extends 'help/base.html' %} {% block breadcrumb_content %}
  • {% link_for _(title), named_route='help.search' %}
  • {% endblock %} {% block help_content %} {% set stats = h.get_site_statistics() %} {% set specimens_url = h.url_for_collection_view() %}

    This is the help page for our Integrated Search feature. Access it here .

    What is integrated search?

    The Data Portal hosts many different datasets. The largest and most commonly used is our Specimens Collection , but there are {{ h.SI_number_span(stats.dataset_count - 1) }} others. Previously, you could only search in one resource (one file within a dataset) at a time, but integrated search allows you to search multiple resources at the same time.

    We've also made the search a lot more powerful by adding advanced features such as boolean logic (AND, OR, and NOT) and number-based range queries (greater than, less than), with more planned in the future.

    The old search

    If you still want to use the old search, we haven't removed anything. Just view the resource you want to search in and everything should be exactly the same as before.

    1. UI Overview

    A labelled screenshot of the whole UI interface.
    Free-text search Search all fields for the specified text.
    Search button Run the current query.
    Helpful buttons (search)
    Help: takes you to this help page.
    Advanced: hides/shows the filters (shown by default).
    Query: shows the JSON query that will be submitted to the API; useful if you need to troubleshoot or want to use the search programmatically.
    Reset: clears all the filters and free-text search.
    Resources: view/edit the list of resources being searched by the current query.
    Helpful buttons (results)
    Cite: generates a DOI for this search.
    Share: generates a memorable link for social sharing.
    Download: download the results.
    Filter terms Filters on specific fields.
    Filter group Groups of filter terms (useful for advanced logic, e.g. OR or NOT queries).
    Result view switcher Change how the results are displayed; defaults to table.
    Results Records matching the current query, displayed using the current view type (e.g. table, list, or gallery).

    2. Constructing a search

    2.1. Selecting resources

    The integrated search can search across multiple resources (individual sets of records within datasets) simultaneously.

    Clicking on Resources will bring up a dialog similar to this one:

    The resources popup dialog, with three datasets shown. Each dataset contains one resource.

    To include a resource in your search, tick the box next to its name. You can select/deselect all the resources within a dataset by clicking the dataset name in green above it, and select/deselect all resources in the list using the "select all" checkbox at the top of the list. To deselect everything except the resources in one particular dataset, hold the alt key and click the dataset name.

    To dismiss the dialog and save your changes, click outside the box.

    2.2. Free-text search

    The free-text search box allows you to search all fields in your chosen resource list. For example, the free-text search "linnaeus" will find records where "Linnaeus" is in scientificName and those where "Linnaeus" is in determinationNames.

    The integrated search UI with 'linnaeus' written in the free-text search field. Six rows of results are shown.

    2.3. Filters

    Below the free-text search is the filter builder. Filter can refer to any part of this: a whole group of multiple items, or a single item. Each of the smallest items are referred to as terms, and they are linked together in groups. For example:

    Two terms (kingdom=animalia and collectionCode=zoo) grouped together in a green box. The words 'all of' are in the top right corner.

    There are two terms in this filter: kingdom = animalia and collectionCode = zoo. They are linked together in an AND (all) group.

    To add a filter term or group, click the icon in the filter builder, then select the option you need.

    The 'add filter' icon when no filters are currently applied. It is just below the free-text search and if prefixed by the text 'ADD FILTERS'.
    or
    The 'add filter' icons inside and outside of a filter group. These are just a plus symbol and have no preceding text.
    then
    The dropdown list of options for a new filter.

    2.3.1. Filter Terms

    A term filters on field names (i.e. it will be applied to every field with that name in every resource in the query - case-insensitively). Terms are the basic units of filtering.

    To add a new term, click the icon and click "new term". This will bring up the term editor.

    The term editor, as it appears when first shown.

    On the left side, select the fields to search in. Typing in the textbox will filter the list; you can then select a field by clicking on it. To remove a field name, click the icon.

    The left side of the term editor, showing a list of fields filtered by the search term 'collection'. 'collectionCode' has already been selected.

    On the right side, specify the details of your query: field type, comparison type, and values.

    A labelled screenshot of the right side of the term editor.

    Field type is the data type that the field will be read as. Most queries will use the default "Text" type, but more complex queries may use the "Number" or "Geo" types. A fourth type, "Any", only has one comparison type ("exists") and just checks if the field has any content at all.

    Comparison type describes how the field value will be compared to the query/comparison value. Each field type has different comparison types available.

    Comparison value is the value to which the field values will be compared.

    Display name is entirely optional, but may be useful for especially complex queries with many terms: it allows you to set a "friendly" display name for the term, so it will be shown in a more memorable way.

    The right side of the term editor, showing a number range query.

    A number range query: matching fields will contain a number between one and five. (Click the angle brackets to toggle equality comparisons).

    The term editor, showing a geo point query.

    A geo point query: matching records will have coordinates within five miles of the specified coordinates.

    NB: geo queries do not use field names. They are based on longitude and latitude points that are set elsewhere by the resource maintainer.

    The term editor, showing a geo polygon query.

    A geo polygon query: matching records will have coordinates inside the specified area.

    You can continue adding as many terms as necessary to your query:

    Three example terms: a text comparison on the field 'collectionCode', a number range comparison on the field 'year', and a term using the display name 'My Complicated Filter'.

    2.3.2. Filter Groups

    Groups allow you to link terms and use some boolean logic in your queries. You can have three types of group: AND (aka 'all of'), OR (aka 'any of'), and NOT (aka 'none of'). To change the group type, click the name in the top left corner. In the groups below, the terms are the same but the group type has changed. This drastically changes the records found.

    An 'AND' filter group, containing two filter terms. The background is green and the text at the top says 'all of'.
    An 'OR' filter group, containing two filter terms. The background is blue and the text at the top says 'any of'.
    A 'NOT' filter group, containing two filter terms. The background is red and the text at the top says 'none of'.
    1. In the first group, each record must match all of the terms; so each record must have collectionCode=BOT and year=2000.
    2. In the second group, each record can match any of the terms; so each record must have collectionCode=BOT or year=2000 (or both!).
    3. In the third group, each record must not match any of the terms; so the records match neither collectionCode=BOT nor year=2000.

    Groups can also be nested under the top level:

    Two 'all of' groups nested within a 'any of' group. 'All of' group one is searching for 'collectionCode = BOT' and 'year = 2000', while 'all of' group 2 is searching for 'collectionCode = PAL' and 'year = 2008'.

    This query would search for either botany records from 2000 or palaeontology records from 2008.

    NB: when you're using the user interface, the whole query is wrapped in a hidden "and" group.

    2.3.3. Presets

    Presets are predefined filter terms that are commonly used and/or need to be dynamically set. There are currently very few of these, but more can be added later. Please {% link_for _('contact us'), named_route='contact.form' %} if you need one of these presets added.

    There are two types of presets: static and dynamic. These are both found in the same place, so the difference is explained here for interest/clarity's sake more than anything else.

    A static preset contains the same information every time, e.g. the Birdwing Butterfly Digitisation will always add a term where project = Birdwing Butterfly Digitisation .

    The term editor for the Birdwing Butterfly Digitisation preset. The project field is selected, the field type is 'text', the comparison type is 'equals', and the comparison value is 'Birdwing Butterfly Digitisation'.

    Dynamic presets change their value dependent on other factors in the query at the time they are added. Once added, they act like every other filter term, so try to add these last if possible. An example is the Has Image preset, which extracts image fields from the current list of resources and adds an "exists" term that checks if these fields have content in them.

    The term editor for the Has Image preset. Two fields are selected: media and associatedMedia. The field type is 'any' and the comparison type is 'exists'.

    In this example, multiple resources with an image field are being searched, so multiple image fields are added to the term. Not every resource will have an image field set, and those that do may share field names with other resources, so the list is likely to be much shorter than the actual list of resources.

    The term editor for the Has Image preset. Only one field is selected: associatedMedia. The field type is 'any' and the comparison type is 'exists'.

    In contrast to the previous image, in this example only one resource with an image field is being searched (or, all the resources being searched have the same image field name), so only one field is selected.

    3. Results

    3.1. Views

    The results of a search can be displayed in different ways, termed "views". To change the view, click the name of the view above the results (see Result View Switcher in 1. UI Overview).

    The fields shown in some views (i.e. Table and List) are dynamically selected via an API method that attempts to guess the "most relevant" fields.

    3.1.1. Table View

    The table view is the default view. It displays the results in tabular form.

    Labelled screenshot of the table results view.

    3.1.2. List View

    The list view displays a short summary of each record individually, along with thumbnails of any associated images (if there are any).

    Clicking on an image brings up the viewer overlay, which displays a larger version and offers an option to download the image.

    Labelled screenshot of the list results view.

    3.1.2. Gallery View

    The gallery view displays all the images associated with records in the search. It does not show records separately, but simply lists out all available images for each page of results.

    Clicking on an image brings up the viewer overlay, which displays a larger version and offers an option to download the image.

    Labelled screenshot of the gallery results view. Labelled screenshot of the image viewer overlay.

    3.2. Buttons

    The three buttons at the top right of the results view: cite, share, and download.

    3.2.1. Cite

    The cite button generates a digital object identifier (DOI) for the search, allowing you to easily cite the results in publications.

    3.2.2. Share

    Similarly to the cite button, the share button also generates a link to the search; but it is intended for more informal use, such as when sharing a search with a colleague. It uses three random words to create a memorable short link (with the option to include the page and view) that can then be shared.

    The 'share' button popup, showing a link to https://data.nhm.ac.uk/search/future-expensive-pelican.

    3.2.3. Download

    The download button allows you to request a full download of the set of results in CSV format. The record data will be automatically packaged up and sent to your email address when ready.

    4. Troubleshooting

    4.1. Resetting

    This is a good place to start for fixing any UI issues, e.g. menus not loading or popup boxes dismissing as soon as you click them.

    1. If you're able to, get a sharing link (see 3.2.2. Share);
    2. Either go to the sharing link (if you have one) or hard-refresh the page with CTRL+F5.
    3. If that still doesn't work, Reset your query, refresh, and start again.

    As always, if you have any further issues, questions, or suggestions, please {% link_for _('contact us'), named_route='contact.form' %}. We are also available on Gitter and Twitter .

    {% endblock %}