Search Objects

Used to control the behaviour how the robot finds the elements you want to automate. These object are then used by the robot to interact with the environment.

Special Objects

class aiviro.core.utils.search_objects.Or(*args: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area])

Used if you need to find one of the specified objects.

Parameters

args – At least two search objects or bounding box areas.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
... # clicks "Log In" or "Sign In" button.
>>> r.click(
...     aiviro.Or(
...         aiviro.Button("Log In"),
...         aiviro.Button("Sign In"),
...     )
... )
>>> r.click(
...     aiviro.Button("Log In") | aiviro.Button("Sign In")
... )
class aiviro.core.utils.search_objects.And(*args: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area])

Used if you want to find all the specified objects. The resulted boxes are concatenated into one list.

Parameters

args – At least two search objects or bounding box areas.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.see(
...     aiviro.Or(
...         aiviro.And(
...             aiviro.Input("Username"),
...             aiviro.Input("Password")
...         ),
...         aiviro.Text('Already logged in')
...     )
... )
>>> r.see(
...     (aiviro.Input("Username") & aiviro.Input("Password"))
...     | aiviro.Text("Already logged in")
... )
class aiviro.core.utils.search_objects.Color(r: int, g: int, b: int, a: float = 1.0)

Used for describing a color which has to be present in element.

Parameters

  • r – red channel <0,256)

  • g – green channel <0,256)

  • b – blue channel <0,256)

  • a – alpha channel <0.0, 1.0>

Example

>>> import aiviro
>>> rb = aiviro.create_desktop_robot()
>>> # check if email field is red when invalid email address
>>> rb.check_elements_color(
...     aiviro.Input("Email"),
...     expected_color=aiviro.Color(255, 0, 0),
...     assert_not=True
... )

See also

see_color() check_elements_color()

class aiviro.core.utils.search_objects.CustomBox(element: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], x_min_offset: Optional[int] = None, y_min_offset: Optional[int] = None, x_max_offset: Optional[int] = None, y_max_offset: Optional[int] = None)

Find an input element on the screen and expand it by selected offset.

Parameters

  • element – Element for which to look for

  • x_min_offset – offset for x_min coordinate of resulted bound_box

  • y_min_offset – offset for y_min coordinate of resulted bound_box

  • x_max_offset – offset for x_max coordinate of resulted bound_box

  • y_max_offset – offset for y_max coordinate of resulted bound_box

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(
...     aiviro.CustomBox(
...         aiviro.Text("Username"),
...         x_min_offset=-10,
...         y_min_offset=30,
...         y_max_offset=30
...     )
... )

See also

Create a custom Search Object, Unrecognized element

class aiviro.core.utils.search_objects.CustomSearchObject(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area])

Used if you want to define your own custom search-object to find mode complex elements.

Parameters

object_to_find – A complex search-object for which to look for.

Example

>>> import aiviro
>>> class PhoneNumbers(aiviro.CustomSearchObject):
...     def __init__(self):
...         super().__init__(
...             aiviro.Below(
...                 aiviro.Text(
...                     r'^\+(\d{1,3})(( ?\d{3}){3})$',
...                     aiviro.find_method.REGEX
...                 ),
...                 aiviro.Text("Phone Numbers")
...             )
...         )
...
>>> r = aiviro.create_desktop_robot()
>>> number_boxes = r.get(PhoneNumbers())

See also

Create a custom Search Object

class aiviro.core.utils.search_objects.BoundaryArea(first_boundary: Optional[Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area]] = None, second_boundary: Optional[Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area]] = None, boundary_type: aiviro.core.constants.ui_constants.BoundaryType = BoundaryType.DEFAULT)

Creates an Area computed from two boundaries. The object is usually used for setting working-area or display-masks.

Parameters

  • first_boundary – First boundary box area, from which the total working area is computed. Can be None but at least one boundary must be set.

  • second_boundary – Second boundary box area, from which the total working area is computed. Can be None but at least one boundary must be set.

  • boundary_type – Method how the working area is calculated. Default calculation creates box encasing both supplied boundary boxes. Other options can be found at BoundaryType

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> with r.add_mask(aiviro.BoundaryArea(aiviro.Text("bound"))):
...     # some code
>>> with r.set_working_area(aiviro.BoundaryArea(aiviro.Text("bound"))):
...     # some code

See also

set_working_area() add_mask()

class aiviro.core.utils.search_objects.LargestSize

It selects the biggest elements based on provided parameters.

Parameters

  • object_to_find – SearchObject from which results the biggest one is selected

  • tolerance – Pixel tolerance for considering several objects as the biggest, respective to the chosen comparison method.

  • element_index – Index of element which should be returned if several, the biggest elements are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> # selects the 2nd biggest text, based on 'width' attribute
>>> largest_width_box = r.get(aiviro.LargestSize.Width(
...     aiviro.Text("text"),
...     tolerance=5,
...     element_index=1
... ))
>>> # selects the biggest button, based on 'area' attribute
>>> largest_area_box = r.get(aiviro.LargestSize.Area(
...     aiviro.Button("OK")
... ))
>>> # selects the biggest input, based on 'height' attribute
>>> largest_height_box = r.get(aiviro.LargestSize.Height(
...     aiviro.Input("")
... ))

Positional Objects

class aiviro.core.utils.search_objects.OnTheLeft(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], reference: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], same_row: bool = True, element_index: Optional[int] = None)

Finding an object to the left of the reference object.

Parameters

  • object_to_find – Element to be searched for

  • reference – Reference element from which the element will be searched

  • same_row – The searched element has to be in the same row as the reference. Default true.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
... # finds element "File" text on the left of "Edit" text
>>> aiviro.OnTheLeft(
...     aiviro.Text("File"),
...     aiviro.Text("Edit")
... )

See also

OnTheRight Above Below

class aiviro.core.utils.search_objects.OnTheRight(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], reference: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], same_row: bool = True, element_index: Optional[int] = None)

Finding an object to the right of the reference object.

Parameters

  • object_to_find – Element to be searched for

  • reference – Reference element from which the element will be searched

  • same_row – The searched element has to be in the same row as the reference. Default true.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
... # finds element "Edit" text on the right of "File" text
>>> aiviro.OnTheRight(
...     aiviro.Text("Edit"),
...     aiviro.Text("File")
... )

See also

OnTheLeft Above Below

class aiviro.core.utils.search_objects.Above(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], reference: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], column_alignment: aiviro.core.utils.search_objects.base.TableAnchor = TableAnchor.LEFT, element_index: Optional[int] = None)

Finding an object above the reference object.

Parameters

  • object_to_find – Element to be searched for

  • reference – Reference element from which the element will be searched

  • column_alignment – Alignment of searched element towards the reference in a column. See TableAnchor for possible options.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
... # finds element "Members" text above of "Settings" text
>>> aiviro.Above(
...     aiviro.Text("Members"),
...     aiviro.Text("Settings")
... )

>>> aiviro.Above(
...     aiviro.Number(),
...     aiviro.Text("ID"),
...     column_alignment=aiviro.table_anchor.RIGHT
... )

See also

Below OnTheLeft OnTheRight

class aiviro.core.utils.search_objects.Below(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], reference: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], column_alignment: aiviro.core.utils.search_objects.base.TableAnchor = TableAnchor.LEFT, element_index: Optional[int] = None)

Finding an object below the reference object.

Parameters

  • object_to_find – Element to be searched for

  • reference – Reference element from which the element will be searched

  • column_alignment – Alignment of searched element towards the reference in a column. See TableAnchor for possible options.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
... # finds element "Settings" text below of "Settings" text
>>> aiviro.Below(
...     aiviro.Text("Settings"),
...     aiviro.Text("Members")
... )

>>> aiviro.Below(
...     aiviro.Number(),
...     aiviro.Text("ID"),
...     column_alignment=aiviro.table_anchor.RIGHT
... )

See also

Above OnTheLeft OnTheRight

class aiviro.core.utils.search_objects.ClosestElement(object_to_find: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area], reference: Union[aiviro.core.utils.search_objects.base.SearchObject, aiviro.core.utils.bound_box.Area])

Finding an object closest to the reference object.

Parameters

  • object_to_find – Element to be searched for

  • reference – Reference element from which the element will be searched

Element Objects

class aiviro.core.utils.search_objects.Text(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, element_index: Optional[int] = None)

Find a text element on the screen.

Parameters

  • label – Text which we are looking for

  • find_method – Method used to find this element. See FindMethod for possible options.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Text("Menu"))
class aiviro.core.utils.search_objects.Input(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find an input element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the input element. If set to false the text element containing the label can be found if the input element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.type_text(aiviro.Input("Username"), "aiviro")
class aiviro.core.utils.search_objects.Button(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find a button element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the button element type. If set to false the text element containing the label can be found if the button element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Button("Log In"))
class aiviro.core.utils.search_objects.CheckBox(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find a checkbox or radiobutton element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the checkbox element type. If set to false the text element containing the label can be found if the checkbox element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.CheckBox("Stay logged in"))
class aiviro.core.utils.search_objects.RadioButton(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find a checkbox or radiobutton element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the RadioButton element type. If set to false the text element containing the label can be found if the RadioButton element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.RadioButton("Female"))
class aiviro.core.utils.search_objects.Toggle(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find a toggle element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the toggle element type. If set to false the text element containing the label can be found if the toggle element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Toggle("Mute sound"))
class aiviro.core.utils.search_objects.TextArea(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find a text area element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the text area element. If set to false the text element containing the label can be found if the text area element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.type_text(aiviro.TextArea("Message"), "This is awesome")
class aiviro.core.utils.search_objects.Icon(label: str = '', find_method: aiviro.core.constants.ui_constants.FindMethod = FindMethod.SIMILAR, strict: bool = False, element_index: Optional[int] = None)

Find an icon element on the screen.

Parameters

  • label – Text identifing this element

  • find_method – Method used to find this element. See FindMethod for possible options.

  • strict – Find only the icon element. If set to false the text element containing the label can be found if the icon element isn’t. Default False

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.click(aiviro.Icon("ARTIN"))
class aiviro.core.utils.search_objects.Number(element_index: Optional[int] = None)

Find a text element which contain only numbers.

Parameters

element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
... # finds number elements in a row of a table
>>> aiviro.OnTheRight(
...     aiviro.Number(),
...     aiviro.Text("table row"),
... )
class aiviro.core.utils.search_objects.Image(path_to_image: Union[str, pathlib.Path], matching_threshold: float = 0.85, element_index: Optional[int] = None)

Find an element on screen using an image of the element you are looking for.

Parameters

  • path_to_image – System path to the image you want to find.

  • matching_threshold – How precise has the image be to find the element. Number 1 is most precise and 0 least precise.

  • element_index – Index of element which should be returned if several same objects are found.

Example

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
... # clicks on a button in incompatible language
>>> r.click(aiviro.Image(
...     "path/to/japanese/button.png",
... ))
class aiviro.core.utils.search_objects.RegexText(regex: str, element_index: Optional[int] = None, find_method: Optional[aiviro.core.constants.ui_constants.FindMethod] = None, force_unicode_search: bool = False)

Alias for Text search-object with REGEX option.

Parameters

  • regex – Regex used to find text elements

  • element_index – Index of element which should be returned if several same objects are found.

  • force_unicode_search – Regex defaults searches in texts normalized to ASCII (without accents etc.) and returns them in this format. If you need to have result with unicode characters set this to True.

Example

>>> import aiviro
>>> aiviro.RegexText(
...     r'\d{2}.\d{2}.\d{4}',
...     element_index=1
... )