Filters

filters are used by multiple hyperportal features, they allow to display/get/modify content depending on set conditions all/any condition with this setting, you can decide whether all conditions should be met or only one example let`s create two conditions for the list view of the contact entity the first condition the country is belgium the second condition gender is female if 'all' is chosen, only contacts that meet two conditions will be displayed if we click on 'meet any of the following conditions' the system will display contacts that meet any of the conditions condition groups the group of conditions allows us to use multiple conditions as one single condition next to the 'add condition' button, there is an 'add group' button instead of one condition, you can add a group of conditions inside the condition group, you can add other conditions or groups group conditions are useful when you want to display/get/modify multiple groups of items for example, i can create a filter that shows me blue diesel cars and red electric cars first i need to create two groups the first group will contain conditions for blue diesel cars, and the second group will contain conditions for red electric cars for the main filter, we should select 'meet any conditions', so the cars from each group will appear we should also select 'meet all' in each group the result of this filter will show two groups of cars "is any of" and "is not any of" operators we have added new filter options, "is any of" and "is not any of" with support for bulk pasting rules these filter operators work with the following property types plain text, entity, phone, email, and link item limit supports up to 1,000 items in the filter if you input more than 1000 items in the field, then the remaining items will be copied to the clipboard if the filter exceeds 1000 values, the "+ add condition" button will be disabled for the safari browser, the "copy" button was added, in other browsers the values that exceed 1000 are automatically added to your clipboard after pasting warning notification popup error notification popup use case example 1\ copy the items from the "country" column in your excel file 2\ in hyperportal, click the "edit view" button 3\ add a filter for the entity property "country" ( \<font color="#ff0000">1\</font> ) and click on the dropdown where you can choose "is any of" the table displayed only items that were added to the filter ( \<font color="#ff0000">2\</font> ) "is not any of" the table displays all elements except those added to the filter ( \<font color="#ff0000">2\</font> ) 4\ paste the copied "country" values into the "search country" field 5\ click the "show result" button to apply the filter dynamic user filter " ⚡️ me (dynamic)" the “me” (dynamic) filter allows dashboards, views, and charts to automatically adapt to the currently logged in user instead of selecting a fixed user (e g , owner = nicolas ), you can configure "owner = me (dynamic)" at runtime, the system resolves “me (dynamic)” to the authenticated user viewing the data this enables a single dashboard or view configuration to be reused by multiple users while still displaying user specific data business impact previously, user specific dashboards required separate configurations such as owner = nicolas owner = ilya this resulted in duplicate dashboards poor scalability with the dynamic user filter , one configuration works for all users scope & limitations the “me” (dynamic) value is available in view filters table filters chart filters dashboard filters the dynamic option is available only for properties where definitionid = contact it can be used exclusively with contact based relational fields (e g , owner) fuzzy search (similar to) fuzzy search is the fuzzy filter operator that finds values spelled with typos, in a different case, without diacritics, or in a different word order use it anywhere resilience to human input errors is needed — from ordinary record search to matching "near duplicates" this document explains what the fuzzy operator gives you and how exactly it works under the hood — with a focus on how a value is split into words/tokens by the analyzer 1\ in brief the fuzzy operator tolerates up to 2 typos per word (edit distance of 2) case and diacritics don't matter mã¼ller = muller = muller , å koda = skoda word order doesn't matter the query jhon doe finds the record doe, john every query word must match (and between words) an extra word narrows the result it is not a substring search berg does not find iceberg matching is against whole tokens (words), not their middle it works on the analyzed (word split) field, not on the "raw" keyword value used by the is / contains / starts with operators 2\ what it does and when to use it the regular operators compare the value as a whole operator what it does is exact match of the whole string (case and diacritics insensitive) contains the value contains a substring starts with / ends with the value starts/ends with a substring fuzzy each query word matches with up to 2 typos, in any order typical scenarios for fuzzy search with typos the user searches for volkwagen , while the database has volkswagen different word order john smith and smith john different spelling/diacritics muller , mã¼ller , mueller detecting "near duplicates" when data is entered by people and minor discrepancies are inevitable fuzzy is intended for text values (names, titles, addresses, e mail) for dates, numbers, and checkboxes use exact operators ( is , comparisons) 3\ how it works under the hood fuzzy search has two sides indexing — how a value stored in a record is split into tokens in advance query — how the string typed by the user is turned into a fuzzy condition matching happens between the value's tokens and the query's words so it is important to understand tokenization first 3 1 indexing splitting a value into tokens (the default analyzer) all text properties ( text , multitext , richtext , file , entity , phonenumber , email ) are indexed in two ways at once { "type" "text", "analyzer" "default", // ← analyzed field the value is split into tokens "fields" { "keyword" { "type" "keyword", "ignore above" 256, "normalizer" "keyword normalizer" // ← raw value as a whole (used by is/contains/…) } } } the propertyid field is analyzed the value is run through the default analyzer and stored as a set of tokens this is what fuzzy works on the propertyid keyword field is the whole value (only folded by case and diacritics via the keyword normalizer ) it is used by is , contains , starts with , ends with , in the default analyzer is a chain of a tokenizer and filters flowchart lr a\["source value"] > b\["icu tokenizer\<br/>split into words"] b > c\["word split\<br/>extra split on hyphens,\<br/>case, digits"] c > d\["icu normalizer\<br/>lowercase,\<br/>unicode normalization"] d > e\["icu folding\<br/>strip diacritics"] e > f\["tokens in the index"] "default" { "type" "custom", "tokenizer" "icu tokenizer", "filter" \["word split", "icu normalizer", "icu folding"] } what each step does icu tokenizer — splits text into words by the unicode text segmentation rules (uax #29) on whitespace, punctuation, and boundaries between different writing scripts works correctly for most languages word split (the word delimiter filter with preserve original true ) — additionally breaks tokens up while keeping the original variant on intra word delimiters wi fi → wi , fi ; on the letter/digit boundary x5 → x , 5 ; on case change (camelcase) iphone → i , phone ; preserve original true — the original token is kept too ( iphone ) icu normalizer — lowercases and performs unicode normalization (a single canonical form of characters) icu folding — strips diacritics and folds characters to their base forms caf㩠→ cafe , mã¼ller → muller , å koda → skoda tokenization examples (illustrative — the exact split is determined by icu rules) field value tokens in the index å koda superb skoda , superb mã¼ller muller iphone iphone , i , phone e mail e , mail bmw x5 bmw , x5 , x , 5 john doe john , doe key idea a value in a record is stored not as one string but as a set of separate word tokens fuzzy search matches the query words against these tokens individually for comparison, the keyword field uses the keyword normalizer — it applies only icu folding (case + diacritics) and does not split the value into words so john doe is stored in keyword as a single john doe 3 2 query how the fuzzy condition is built the string entered by the user is turned into a fuzzy condition as follows an empty value ( null , undefined , empty string) → filter invalid error the input is trimmed and split on whitespace ( \s+ ) into words repeated and surrounding spaces are collapsed each word becomes a term field\ word 2 it targets the analyzed field, not keyword ; the 2 suffix is lucene fuzzy search syntax edit distance 2 ; special characters are escaped before the 2 suffix is appended if there are several words, they are combined with and and wrapped in parentheses conversion examples user input generated condition jhon name\ jhon 2 jhon doe (name\ jhon 2 and name\ doe 2) foo bar baz (name\ foo 2 and name\ bar 2 and name\ baz 2) jhon doe (name\ jhon 2 and name\ doe 2) (spaces collapsed) jh\ on name\ jh\\\ on 2 (colon escaped) why this gives "word order independence" each query word is searched among any of the field's tokens independently, and and requires that all words are found order plays no role 3 3 what "edit distance 2" means 2 is the maximum damerau–levenshtein distance between a query word and a token in the index up to 2 operations per word are allowed, where one operation is inserting one character ( jon → john ), deleting one character ( johnn → john ), substituting one character ( jahn → john ), transposing two adjacent characters ( jhon → john ) 2 is the maximum value that lucene fuzzy search supports the query word also passes through the field analyzer before matching, so case and diacritics in the query don't matter either 4\ examples what matches and what doesn't a record with the name property value = john doe → tokens john , doe query ( fuzzy ) matches? why jhon ✅ jhon → john transposition = 1 operation (≤ 2) jon ✅ inserting h = 1 operation jhon doe ✅ both words matched (and), order doesn't matter doe john ✅ word order doesn't matter john smith ✠smith matched no token → and fails berg (for the value iceberg ) ✠not a substring berg → iceberg = 3 operations diacritics and case the record mã¼ller (token muller ) matches the queries muller , muller , mãœller , mueller (the last one — 1 deletion) 5\ limitations and pitfalls no more than 2 typos per word a heavily distorted word ( volswgn instead of volkswagen ) won't match short words cause false positives with distance 2, the word ford matches word , form , fort , food , and so on for short values (part numbers, 3–4 character codes) fuzzy search is best avoided it is not a substring search for "contains a fragment" use contains and between words every extra word in the query narrows the result — all words must match numbers and punctuation are split by the word split filter, so behavior on codes, part numbers, and skus may be surprising — prefer is / contains empty value is not allowed an empty string leads to a filter invalid error