Tips & Tricks

Do you want to know how to make your Aiviro scripts faster, cleaner, better and more reliable? Then you’ve come to the right place. These tips & tricks are made to help you with that!

Search object strategy of Aiviro

We’re using the concepts of Element Objects for specifying which object you can interact with. In short, if you want to click on a Button with the text “Search”, you can do it the following way:

>>> import aiviro
>>> r = aiviro.create_web_robot(headless=True)
>>> r.click(aiviro.Button("Search"))

Aiviro will take care of everything. It’ll recognize all the elements (Button, Input, Text, etc.) present on the screen. And it’ll search for the Button which contains the text “Search”.

Let’s look deeper into how exactly Aiviro is selecting the right element for you. This will help you to write fast and reliable scripts. There are two main categories of elements that we recognize:

Tip

In case of searching for Text element, Aiviro will search through for the corresponding text in all of Text elements.

Tip

Every one of Input, Button, CheckBox, RadioButton, Toggle or TextArea (Icon excluded) elements are identified by some text. It’s mostly a text which is inside or very close to the corresponding element. This text is used in element lookup.

Let’s take the Button from the example above Button(“Search”). Aiviro will firstly look through all of the buttons. If no Button is found, then it will search through the other Predict elements (Icon excluded). In some scenarios searching for Button can result in finding an Input.

If still none of the searches resulted in success then Aiviro will search through all Text elements.

Tip

When searching for Icon element, Aiviro will firstly look through all icons. If the icon isn’t found then it will search through all of the Text elements.

Note

Every Element Objects contain an argument strict: bool. If it’s set to True, Aiviro will search only for the specified element type.

They’re also several options of matching the text which specifies the element. See Control Parameters for more info.

>>> r.click(aiviro.Button(
...     "Search",
...     strict=True,
...     find_method=aiviro.find_method.EQUAL
... ))

Create a working area from a newly opened window

Having modal windows for uploading files or changing options is fairly common in applications nowadays. Also, applications needn’t have to be opened in fullscreen mode as we’ll demonstrate in our example below.

In these types of situations we want to work only inside of this newly opened window, but how to do it? For these purposes, we have implemented an option to create a working area from the difference of two screenshots. In our example, we’ll open the recycle bin window. Let’s see how to do it.

>>> import aiviro
... # The robot will connect to a remote windows machine
>>> r = aiviro.create_rdp_robot("ip-address", "username", "password", "domain")
... # We'll wait until the machine is fully loaded by waiting until icon for a Recycle Bin icon
>>> bin_box = r.wait_for(aiviro.Icon("Recycle Bin"))
... # We'll create a checkpoint for our working area,
... # this command will make screenshot of the display and store it by a specified name ("desktop")
>>> r.add_working_area_checkpoint("desktop")
... # Open recycle bin by double-clicking on it
>>> r.double_click(aiviro.Icon("Recycle Bin"))
... # Wait until Recycle Bin window is open
>>> r.wait_for(aiviro.Text("This PC"))
... # This command will make another screenshot and compare it with the previous one from our checkpoint ("desktop")
... # the difference between the screens is our working area.
>>> with r.set_working_area_by_checkpoint("desktop"):
...     # now we'll click on a "Recycle Bin" text which is inside of our opened window
>>>     r.click(aiviro.Text("Recycle Bin"))
_images/working-area_win_empty.png

Loaded Windows desktop

_images/working-area_win_area.png

Created working area using checkpoint commands

Create a custom Search Object

Sometimes you want to find a specific element or specific text and you want to do that several times in your script. In that case it comes handy to create your own, custom Search Object. The concept of Search Objects should already be familiar to you, as it’s the basic block for working with Aiviro. If not please check it out before reading this.

There are two ways how to create a custom Search Object:

First one comes handy if you are looking for a specific text, let’s say a date (e.g. 29.02.2020). You can create a search-object called Date() which inherits from an search-object Text, as follows:

class Date(aiviro.Text):
    def __init__(self, element_index: int = None)
        super().__init__(
            label=r'\d{2}.\d{2}.\d{4}',
            find_method=aiviro.find_method.REGEX,
            element_index=element_index
        )

We also included argument element_index into element’s constructor, to give us a more robust option how to work with our custom search-object.

The second option for creating your custom search-object is useful, if you are looking for a more complex object on the screen. Let us show an example where we want to get all phone numbers from the table’s column:

class PhoneNumbers(aiviro.CustomSearchObject):
    def __init__(self):
        super().__init__(
            aiviro.Below(
                aiviro.Text(
                    r'^\+(\d{1,3})(( ?\d{3}){3})$',
                    aiviro.find_method.REGEX
                ),
                aiviro.Text("Phone Numbers")
            )
        )

After creating our custom search-objects, we can now easily use them in our script.

>>> import aiviro
>>> from some_module import Date, PhoneNumbers
>>> r = aiviro.create_desktop_robot()
>>> r.click(Date())
>>> phone_boxes = r.get(PhoneNumbers())

Unrecognized element

In a few cases, when we use Aiviro for an app for which it has never been used before, it may not recognize all the elements on the screen. Like in the example below, where Input element is not recognized.

_images/input_unrecognized.png

Unrecognized Input element

To interact and type text into the unrecognized Input element, we can customize the Text("POPIS/POZNAMKA") with a CustomBox search-object.

>>> import aiviro
>>> r = aiviro.create_desktop_robot()
>>> r.type_text(
...     aiviro.CustomBox(
...         aiviro.Text("POPIS/POZNAMKA"),
...         y_min_offset=35,
...         y_max_offset=35
...     ),
...     "text to type"
... )

Exceptions

Too many elements found

Another exception can be raised if there are several same elements on the screen. For example, we want to click on the element Text("HTML"), but there is 14 of them on the screen.

>>> import aiviro
>>> r = aiviro.create_web_robot(headless=True)
>>> r.go_to_url("https://developer.mozilla.org/en-US/docs/Web/HTML")
>>> r.click(aiviro.Text("HTML"))

Traceback (most recent call last):
  File "/home/*/sandbox/demo_test.py", line 33, in <module>
    r.click(aiviro.Text("HTML"))
  File "/home/*/aiviro/robot.py", line 408, in click
    move_away=move_away,
  File "/home/*/aiviro/utils/common.py", line 74, in inner
    x = foo(*args, **kwargs)
  File "/home/*/aiviro/commands/commands_device.py", line 44, in execute
    stable_screen, (box,) = self.ser_fact.cross_val_service.find_and_validate_search_objects([element])
  File "/home/*/aiviro/services/crossvalidation_service.py", line 208, in find_and_validate_search_objects
    elements=elements, allow_list_of_boxes=False
  File "/home/*/aiviro/services/crossvalidation_service.py", line 162, in find_and_validate_search_objects_as_list
    find_many_boxes=allow_list_of_boxes,
  File "/home/*/aiviro/services/find_service/find_service.py", line 82, in process_search_object
    f"Too many elements ({len(boxes)}) found for {element}."
aiviro.utils.exceptions.SearchObjectError: Too many elements (14) found for <Text "HTML" FindMethod.SIMILAR/>.

The solution is to specify element_index in the Text("HTML") element, or set-up some working area.

>>> import aiviro
>>> r = aiviro.create_web_robot(headless=True)
>>> r.go_to_url("https://developer.mozilla.org/en-US/docs/Web/HTML")
... # we will click on the first "HTML" element using 'element_index' argument
>>> r.click(aiviro.Text("HTML", element_index=0))
... # or set-up working area
>>> with r.set_working_area_by_boundaries(
>>>     second_boundary=aiviro.Text('Change language')
>>> ):
>>>     r.click(aiviro.Text('HTML'))