Search Objects

SearchObjects are fundamental elements in Aiviro for identifying and automating interactions with screen elements. These objects allow the robot to find and interact with elements on the screen efficiently.

Special Objects

class aiviro.core.utils.search_objects.Or(*args: SearchObject | Area)

Used to find one of the specified objects.

Parameters:: args – At least two search objects or bounding box areas.
Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
... # clicks "Log In" or "Sign In" button.
>>> r.click(
...     aiviro.Or(
...         aiviro.Button("Log In"),
...         aiviro.Button("Sign In"),
...     )
... )
>>> r.click(
...     aiviro.Button("Log In") | aiviro.Button("Sign In")
... )

class aiviro.core.utils.search_objects.And(*args: SearchObject | Area)

Used to find all the specified objects. The resulting boxes are concatenated into one list.

Parameters:: args – At least two search objects or bounding box areas.
Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.see(
...     aiviro.Or(
...         aiviro.And(
...             aiviro.Input("Username"),
...             aiviro.Input("Password")
...         ),
...         aiviro.Text('Already logged in')
...     )
... )
>>> r.see(
...     (aiviro.Input("Username") & aiviro.Input("Password"))
...     | aiviro.Text("Already logged in")
... )

class aiviro.core.utils.search_objects.Color(r: int, g: int, b: int, a: float = 1.0)

Used for describing a color presented in an element.

Parameters:

r – red channel (0 - 255)
g – green channel (0 - 255)
b – blue channel (0 - 255)
a – alpha channel (0.0 - 1.0)

Example:

>>> import aiviro
>>> rb = aiviro.create_desktop_robot()
>>> # Check if the email field is red when the email address is invalid.
>>> rb.check_elements_color(
...     aiviro.Input("Email"),
...     expected_color=aiviro.Color(255, 0, 0),
...     assert_not=True
... )

See also

see_color() check_elements_color()

Find an input element on the screen and expand it by selected offset.

Parameters:

element – Element to look for
x_min_offset – offset for x_min coordinate of the resulting bound_box.
y_min_offset – offset for y_min coordinate of the resulting bound_box.
x_max_offset – offset for x_max coordinate of the resulting bound_box.
y_max_offset – offset for y_max coordinate of the resulting bound_box.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(
...     aiviro.CustomBox(
...         aiviro.Text("Username"),
...         x_min_offset=-10,
...         y_min_offset=30,
...         y_max_offset=30
...     )
... )

class aiviro.core.utils.search_objects.CustomSearchObject(object_to_find: SearchObject | Area)

Used to define your own custom search object for finding complex elements.

Parameters:: object_to_find – A complex search object to look for.
Example:

>>> import aiviro
>>> class PhoneNumbers(aiviro.CustomSearchObject):
...     def __init__(self):
...         super().__init__(
...             aiviro.Below(
...                 aiviro.Text(
...                     r'^\+(\d{1,3})(( ?\d{3}){3})$',
...                     aiviro.find_method.REGEX
...                 ),
...                 aiviro.Text("Phone Numbers")
...             )
...         )
...
>>> r = aiviro.create_desktop_robot()
>>> number_boxes = r.get(PhoneNumbers())

See also

Create a custom Search Object

class aiviro.core.utils.search_objects.BoundaryArea(first_boundary: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area | None = None, second_boundary: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area | None = None, boundary_type: ~aiviro.core.constants.ui_constants.BoundaryType = <BoundaryType.DEFAULT: 1>)

Creates an area computed from two boundaries, typically used for setting working areas or display masks.

Parameters:

first_boundary – First boundary box area to compute the total working area. Can be None, but at least one boundary must be set.
second_boundary – Second boundary box area to compute the total working area. Can be None, but at least one boundary must be set.
boundary_type – Method to calculate the working area. Default creates a box encasing both supplied boundary boxes. Other options can be found in BoundaryType

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> with r.add_mask(aiviro.BoundaryArea(aiviro.Text("bound"))):
...     # some code
>>> with r.set_working_area(aiviro.BoundaryArea(aiviro.Text("bound"))):
...     # some code

See also

set_working_area() add_mask()

class aiviro.core.utils.search_objects.LargestSize

Selects the largest elements based on provided parameters.

Parameters:

object_to_find – SearchObject from which the largest result is selected.
tolerance – Pixel tolerance for considering several objects as the largest.
element_index – Index of the element to return if several largest elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> # selects the 2nd biggest text, based on 'width' attribute
>>> largest_width_box = r.get(aiviro.LargestSize.Width(
...     aiviro.Text("text"),
...     tolerance=5,
...     element_index=1
... ))
>>> # selects the biggest button, based on 'area' attribute
>>> largest_area_box = r.get(aiviro.LargestSize.Area(
...     aiviro.Button("OK")
... ))
>>> # selects the biggest input, based on 'height' attribute
>>> largest_height_box = r.get(aiviro.LargestSize.Height(
...     aiviro.Input("")
... ))

Positional Objects

class aiviro.core.utils.search_objects.OnTheLeft(object_to_find: SearchObject | Area, reference: SearchObject | Area, same_row: bool = True, element_index: int | None = None)

Finds an object to the left of the reference object.

Parameters:

object_to_find – Element to search for.
reference – The reference element from which the search is conducted.
same_row – Whether the searched element must be in the same row as the reference. Default is True.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
... # finds element "File" text on the left of "Edit" text
>>> aiviro.OnTheLeft(
...     aiviro.Text("File"),
...     aiviro.Text("Edit")
... )

See also

OnTheRight Above Below

class aiviro.core.utils.search_objects.OnTheRight(object_to_find: SearchObject | Area, reference: SearchObject | Area, same_row: bool = True, element_index: int | None = None)

Finds an object to the right of the reference object.

Parameters:

object_to_find – Element to search for.
reference – The reference element from which the search is conducted.
same_row – Whether the searched element must be in the same row as the reference. Default is True.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
... # finds element "Edit" text on the right of "File" text
>>> aiviro.OnTheRight(
...     aiviro.Text("Edit"),
...     aiviro.Text("File")
... )

See also

OnTheLeft Above Below

class aiviro.core.utils.search_objects.Above(object_to_find: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area, reference: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area, column_alignment: ~aiviro.core.utils.search_objects.base.TableAnchor = <TableAnchor.LEFT: 1>, element_index: int | None = None)

Find an object above a reference object.

Parameters:

object_to_find – Element to search for.
reference – The reference element from which the search is conducted.
column_alignment – The alignment of the searched element relative to the reference in a column. See TableAnchor for options.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
... # finds element "Members" text above of "Settings" text
>>> aiviro.Above(
...     aiviro.Text("Members"),
...     aiviro.Text("Settings")
... )

>>> aiviro.Above(
...     aiviro.Number(),
...     aiviro.Text("ID"),
...     column_alignment=aiviro.table_anchor.RIGHT
... )

See also

Below OnTheLeft OnTheRight

class aiviro.core.utils.search_objects.Below(object_to_find: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area, reference: ~aiviro.core.utils.search_objects.base.SearchObject | ~aiviro.core.utils.bound_box.Area, column_alignment: ~aiviro.core.utils.search_objects.base.TableAnchor = <TableAnchor.LEFT: 1>, element_index: int | None = None)

Finding an object below the reference object.

Parameters:

object_to_find – Element to search from
reference – The reference element from which the search is conducted.
column_alignment – The alignment of the searched element relative to the reference in a column. See TableAnchor for options.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
... # finds element "Settings" text below of "Members" text
>>> aiviro.Below(
...     aiviro.Text("Settings"),
...     aiviro.Text("Members")
... )

>>> aiviro.Below(
...     aiviro.Number(),
...     aiviro.Text("ID"),
...     column_alignment=aiviro.table_anchor.RIGHT
... )

See also

Above OnTheLeft OnTheRight

class aiviro.core.utils.search_objects.ClosestElement(object_to_find: SearchObject | Area, reference: SearchObject | Area)

Find the object closest to the reference object.

Parameters:

object_to_find – Element to search for.
reference – The reference element from which the search is conducted.

Element Objects

class aiviro.core.utils.search_objects.Text(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, element_index: int | None = None)

Find a text element on the screen.

Parameters:

label – The text to look for.
find_method – Method used to find the element. See FindMethod for options.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Text("Menu"))

class aiviro.core.utils.search_objects.Input(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find an input element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.type_text(aiviro.Input("Username"), "aiviro")

class aiviro.core.utils.search_objects.Button(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find a button element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Button("Log In"))

class aiviro.core.utils.search_objects.CheckBox(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find a checkbox element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.CheckBox("Stay logged in"))

class aiviro.core.utils.search_objects.RadioButton(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find a radiobutton element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.RadioButton("Female"))

class aiviro.core.utils.search_objects.Toggle(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find a toggle element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Toggle("Mute sound"))

class aiviro.core.utils.search_objects.TextArea(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find a text-area element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.type_text(aiviro.TextArea("Message"), "This is awesome")

class aiviro.core.utils.search_objects.Icon(label: str = '', find_method: ~aiviro.core.constants.ui_constants.FindMethod = <FindMethod.AUTO: 1>, strict: bool = False, element_index: int | None = None)

Find an icon element on the screen.

Parameters:

label – The text identifying this element.
find_method – Method used to find the element. See FindMethod for options.
strict – If set to False, the text element containing the label will be found if the input element isn’t. Default is False.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Icon("ARTIN"))

class aiviro.core.utils.search_objects.Number(element_index: int | None = None)

Find a text element which contains only numbers.

Parameters:: element_index – The index of the element to return if multiple matching elements are found.
Example:

>>> import aiviro
... # finds number elements in a row of a table
>>> aiviro.OnTheRight(
...     aiviro.Number(),
...     aiviro.Text("table row"),
... )

class aiviro.core.utils.search_objects.Image(path_to_image: str | Path | ndarray, matching_threshold: float = 0.85, element_index: int | None = None)

Find an element on screen using an image of the element you are looking for.

Parameters:

path_to_image – Path to the image file or numpy array of the image.
matching_threshold – The precision required to find the element. A value of 1 is most precise, and 0 is least precise.
element_index – The index of the element to return if multiple matching elements are found.

Example:

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
... # clicks on a button in incompatible language
>>> r.click(aiviro.Image(
...     "path/to/japanese/button.png",
... ))

class aiviro.core.utils.search_objects.RegexText(regex: str, element_index: int | None = None, find_method: FindMethod | None = None, force_unicode_search: bool = False)

Alias for Text search-object with REGEX option.

Parameters:

regex – Regex pattern used to find text elements.
element_index – The index of the element to return if multiple matching elements are found.
force_unicode_search – Regex defaults searches in texts normalized to ASCII (without accents etc.) and returns them in this format. If you need to have result with unicode characters set this to True.

Example:

>>> import aiviro
>>> aiviro.RegexText(
...     r'\d{2}.\d{2}.\d{4}',
...     element_index=1
... )