Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog.

3.13.1

Released on 13-09-2023

Added

  • flag docker-compose into deployment commands to use docker-compose command

  • execute_script command into WebRobot

  • raise of RuntimeError when robot was closed and user tries to use it

Changed

  • by default docker compose (plugin version) is used in aiviro-cli deployment commands

  • cumulative enum implementation to be an extension of python builtin enum type

Fixed

  • loading of aiviro-config in flows

  • ci pipeline for prefect-agent build

  • loading of yaml and aiviro configuration files correctly from old environment variables

  • converting of total-amount string to Decimal in UniversalParser

3.13.0

Released on 23-08-2023

Added

  • public method clean for every context-manager to clean up after itself

  • ErrorHandler for saving and processing image & text logs when error occurs

  • aiviro-cli test-connection commands for testing IMAP & SMTP server configurations

  • unencrypted IMAP mailbox support

  • parsing of bank-account information into UniversalParser

  • parsing of due-date information into UniversalParser

  • parsing of company-name, based on IČO, into UniversalParser

  • separate reader analysis for OCR and Digital pdf documents

  • server and agent deploy command into aiviro-cli deploy

  • deployment section into documentation

  • docker build into CI for prefect agent & ui

  • error handling for invalid metadata-log records, skip them

  • type restriction, arguments of WaitUntilDisappear can only be of type SearchObject

  • aiviro-cli config command to set-up api credentials and other settings

  • delete_folder method into EmailClient class

Changed

  • context-manager logic from decorator into separate class

  • loading strategy of aiviro-config files, several options are now available

  • alignment of boxes in one line, added threshold for which the alignment is ignored

Fixed

  • context-manager for setting working-area while other context-manager is active

  • qr-code decoding for empty values

  • pip dependency backtracking issue

  • empty string argument in Notifier message object

  • adding text into PDF when exception occurs

  • recursion bug when using BaseKey in interactive console

  • empty file processing in EmailExtractor

  • rendering of email’s report for desktop Outlook

  • attribute access in cache-file if attribute does not exist

Deprecated

  • environment variable “AIVIRO_CONFIG”

Removed

  • deprecated email-client methods setup_imap and setup_smtp

3.12.0

Released on 03-07-2023

Added

  • bigger retry-waiting time for sending email

  • standard BoundBox output for UniversalParser lookup objects

  • post-processing method for UniversalParser lookup objects

  • auto-rotation functionality of scanned documents into PDFRobot

  • outlook authentication token refresh, if 401 is returned

  • merge functionality for YAMLConfig

  • wait_until_ready method for RDPRobot

  • BaseScenario object that helps with initialization of some basic Aiviro services

  • support for custom split-condition argument in document items detection

  • no-verify argument for ssl certificate verification in SMTP & IMAP basic-auth email methods

  • dynamic repack of the cache to reduce its disk-size

Changed

  • default endpoint from “api.*.aiviro.com” to “api.aiviro.com” in doc & config file

Fixed

  • pSKeyword error while loading pdf-file

  • handling of ReadTimeout exception for api-calls

  • adding text to an empty scanned PDF file

  • improved accuracy of total-amount extraction

  • recursive loading of Outlook email attachments

  • streaming initialization error

  • handling of locked cache file

  • pDFRobot closing in global-context object

  • dynamic removal of cache records (time reduction)

Removed

  • old request structure to api.aiviro.com endpoints

3.11.2

Released on 10-05-2023

Fixed

  • encoding of PDF attachments

  • position recalculation when adding text or image into PDF file

3.11.1

Released on 03-05-2023

Added

  • aiviro-cli deploy command to deploy certain Aiviro functionality as a web application

  • exception handling in global-context, when service cannot be closed properly

Fixed

  • read error of non-existing temporary folder

3.11.0

Released on 25-04-2023

Added

  • extraction of attachments & comments from PDF file

  • new properties attachments and comments into PDFRobot

  • documents item parsing into UniversalParser

  • lazy loading of PDF files for text or image addition

  • oneDrive module which allows to perform basic operations on user, shared and sites drives

  • functionality to add new-column in CSVWriter

  • excel module to read .xlsx files

  • documentation examples for NotifierManager

  • test into CI for default dependency imports

  • inactive option (gray color) for notifier

Changed

  • dependency from PyPDF4 to pypdf

  • longer waiting/retry time for api requests (up to 5 minutes in total)

  • CSVWriter can be initialized also with empty PDFRobot

  • documentation for EmailNotifier

Fixed

  • loading of PDF files with invalid structure

  • improved order-number extraction in universal-parser

  • warning message about package versions in aiviro-cli log command

  • closing of streaming service when communication pipe between processes is broken

  • long compression time of complex png images

3.10.0

Released on 14-03-2023

Added

  • pause & unpause methods into robot

  • get_last_image method into ImageCollector service

  • unit-tests for runner-service

  • __add__ method for CumulativeEnum object, used in BaseKey

  • activate robot method into dynamic robots

  • missing metrics into global-context

Changed

  • code structure of how streaming is started

Fixed

  • bug with token normalization replacing wrong elements

  • initialization of RDP shared-folder, when only path is provided without path-name

3.9.2

Released on 02-03-2023

Fixed

  • PDFRobot failing to read some pdfs

  • PDFRobot spliting words at some unicode chars

  • IndexError exception while parsing total_amount

3.9.1

Released on 28-02-2023

Added

  • documentation for universal-parser module

  • new version of Universal Parser

  • additional tests for universal-parser

  • improve universal-parser performance

3.9.0

Released on 13-02-2023

Added

  • access to created working-area & display masks by with statement

  • last_block variable into NotifierManager

  • send_report method in NotifierManager to send messages across all notifiers based on scope

  • oauth2 authentication for SMTP & IMAP into EmailClient

  • .coveragerc file to improve code coverage

  • evaluation test for universal-parser

  • global-context object that contains information about script & correctly closes robots & services

  • additional metadata into aiviro-metrics

  • method as_file_attachment to convert IMAPMessage into IMAPAttachment

  • support for processing IMAPAttachment in send_email method

  • point into image-logs for click commands

  • unit-tests for desktop, rdp & web devices

  • additional methods for easier PDF annotation

  • new search-object Biggest

  • argument ignore_exception into QR code decoder, to ignore invalid codes

  • text extraction from body of HTML email content into email.text variable

  • renaming of aiviro processes (on linux only) for better debugging

Changed

  • updated project metadata information

  • documentation for Email’s Outlook OAuth2

Fixed

  • logic for selecting correct working-area by checkpoints

  • default parameter of send_report in EmailNotifier

Removed

  • support for python3.7

3.8.0

Released on 29-12-2022

Added

  • command add_mask_windows_taskbar to cover windows taskbar

  • strict argument into check_are_intersecting command

  • NotifierManager to share messages between several notifiers at once

  • powershell command create_and_copy_powershell_script

  • aiviro-cli command reader find to validate if exported data is present in pdf-files

  • test coverage reports into GitLab-CI

  • negative results into cache

  • dynamic records removal from cache

  • option FindMethod.GLOBAL which finds all occurrences of searches not only the first one

  • methods for text and image addition to the PDFRobot

  • method for saving the PDF file, modified or not, was added to the PDFRobot

Changed

  • imports of notifier components from aiviro.modules.email.notifier to aiviro.modules.notifier

Fixed

  • typing of all functions to follow the newest versions of mypy

  • error some pdfs failing on decoding error

  • regex backtracking error

Deprecated

  • creation of message directly through EmailNotifier, use instead NotifierManager

  • recipients argument in send_report() method in EmailNotifier, moved into object’s constructor

3.7.1

Released on 01-12-2022

Added

  • info log into pdf-robot about the name of parsed pdf-file

  • Secret object for better handling of secret/sensitive texts

  • logging support for new pyfreerdp version

Changed

  • all exceptions which are triggered from worker process now use ChildProcessError class

Fixed

  • added timeout handling for starting worker process

3.7.0

Released on 29-11-2022

Added

  • argument auto_move into EmailExtractor, extract_* methods

  • email argument into EmailExtractor Conditions

  • EmailDatetimeCondition & EmailSubjectCondition new EmailExtractor conditions

  • supported python, os & cpu versions into documentation

  • new version of API calls to backend-vision server

  • api versioning, new endpoint /v1/detect

  • aiviro-cli reader commands - stats & extract

  • search-object ClosestElement to find closest object to its reference

  • simple scenario metrics calculation, cache usage, pdfs, api calls

  • checkout to master/develop into flow templates

  • logging of INFO logs into separate text-file

Changed

  • pillow version to 9 so higher versions of python can use prebuilt wheels

  • clean-up internal code of Universal Parser

Fixed

  • report template for microsoft Outlook

Deprecated

  • old configuration of backend-vision server

3.6.0

Released on 24-10-2022

Added

  • find_method variable instead of text_find_methods

  • chaining capability for key, find_method & boundary_type

  • processing of Outlook itemAttachments, as if attachment is additional e-mail

  • MultiPDFRobot, can extract and hold data of several PDF files

  • variable IS_RUNNING_IN_PREFECT into aiviro package

  • get_working_area command

Changed

  • some headless tests into static-robot, to speed them up

  • version of imap-tools dependency from 0.49.1 to 0.57.0

  • file & inline email-attachments into file_attachments variable

  • name of method do_text_extraction to parse in PDF robot

Fixed

  • visibility of Outlook OAuth2 folders

Deprecated

  • variable text_find_methods

Removed

  • argument skip_chars argument from PDF robot, moved into parse method

3.5.2

Released on 21-09-2022

Fixed

  • data-type of shared-folder path in RDP robot

3.5.1

Released on 19-09-2022

Added

  • group argument into mask_* commands to add/delete only specific group of masks

Fixed

  • ci deploy job now working correctly

  • pdf extraction error which sometimes didn’t split words correctly

3.5.0

Released on 15-09-2022

Added

  • methods for Outlook with OAuth2.0 & graf.microsoft endpoints into EmailClient

  • setup_imap_basic_auth & setup_smtp_basic_auth methods in EmailClient

  • unit testing for multiple versions of python (3.7, 3.8, 3.9, 3.10)

Changed

  • updated versions of dependencies

Fixed

  • pdf text duplication issue

Deprecated

  • username & password arguments in EmailClient

  • setup_imap & setup_smtp methods in EmailClient

3.4.0

Released on 30-08-2022

Added

  • czech Invoice-Payment decoder for QR codes (https://qr-faktura.cz/)

  • CheckCommandError exception for check_* commands

  • documentation section for aiviro-exceptions

  • implicit_wait option into user_config options

  • alternatives for some standard fonts in pdf

  • support for pdfs which don’t have characters in top level

  • Copy-Item command into PowerShell

  • get_shared_folder_path variable into PowerShell to get path on remote machine

  • universal_parser modul

  • exception_to_raise argument into get, see & wait_* commands

  • create command into aiviro-cli to initialize Aiviro project structure & add scenarios

Changed

  • internal use of os.path to pathlib.Path

  • internal points distance calculation to use math module

  • name of FindService to SearchObjectService

  • names of Positioned* to Positional*

  • location of robots from one big script into separate ones

  • class Robot, split into BaseRobot, DynamicRobot & StaticRobot

  • internal use of deque for OrderedDict

Fixed

  • pdf robot font parsing

  • pdf character merging not working sometimes properly

Removed

  • decoder argument from CodesExtractor

3.3.1

Released on 25-07-2022

Changed

  • regex dot changed to new style

3.3.0

Released on 25-07-2022

Added

  • aiviro-cli log analyze command

  • option to edit Notification type in EmailNotifier - notification_type variable

  • argument email_attachments into EmailNotifier

  • user_config object to change similarity_threshold & set custom text-normalization

  • user Configuration section into documentation

  • AutoSelectParsingStrategy for ParserHandler

  • ignorable_parser_objects argument into ParserHandler

  • parser attribute into ParserProcessingError

  • forward method into email module

  • encoding argument into CSV & YAML configurations

Changed

  • aiviro-debugger into aiviro-cli

  • functionality of *_by_boundary commands into BoundaryArea search-object

Fixed

  • documentation typos in email-notifier section

  • texts & examples in documentation

  • loading of duplicated parser-classes

Deprecated

  • command add_mask_by_boundaries

  • command add_mask_by_margin

  • command set_working_area_by_boundaries

Removed

  • parser_name attribute from ParserProcessingError

3.2.4

Released on 20-07-2022

Added

  • missing keys into web-robot

Fixed

  • bug when creating pdf robot state data

3.2.3

Released on 19-07-2022

Fixed

  • regex issue when using escaped special character

3.2.2

Released on 11-07-2022

Added

  • y-alignment of ocr-boxes when merged into common line

  • regex support for backref characters

Changed

  • added normalized_text of BoundBox to noncomparable keys as it is lazy evaluated

Fixed

  • ocr-boxes line-grouping into fix order

  • bound box mutable issue

  • regex normalization for charclasses containing space

3.2.1

Released on 01-07-2022

Added

  • current_page property into PDFRobot

  • is_digital property into PDFRobot

Changed

  • opencv-python-headless dependency back to opencv-python

Fixed

  • typo in unable-to-process message in EmailNotifier

  • ocr-box connection for bigger boxes

  • cache version incompatibility, between <3.1.5 & >=3.1.5

3.2.0

Released on 29-06-2022

Added

  • pdf robot now uses masks correctly

  • commonly used Powershell commands for RDP-Robot

  • subject argument for sending email-notification

  • last_block property into EmailExtractor

  • option to pass Msg or str argument as message for EmailExtractor

  • Item object for creating notification messages

  • duration tag into EmailNotifier

  • functionality to load aiviro_config from env-variable in prefect flows

  • configuration loaders for YAML & CSV files

  • ignore_exception option to get element

  • emailExtractor for simplified way of email & attachment extraction

  • documentation for image-buffer

  • default values into doc for Aiviro configuration file

  • skip_chars argument into create_pdf_robot method

  • codes module for reading QR & Bar codes from image

Changed

  • regex normalization to be more intelligent and prevent normalization of regexes with catastrophic backtracking

  • opencv-python dependency to opencv-python-headless

Fixed

  • timezone in EmailNotifier & message typo

  • less verbose log messages when creating pdf robot

  • duplicated OCR boxes in metadata

  • box sorting for ocr-line-grouping

  • bug with memory not releasing before application exit

  • combination of paste & blocking argument in transfer_files_to_guests_clipboard cmd

Removed

  • sorting of email notifications, make it optional

  • force option for robot-streaming

3.1.5

Released on 09-06-2022

Added

  • check when file exists when using RDP transfer to host

Changed

  • pip project summary, email, licence

Fixed

  • reference box filtering in positional elements

  • wrong argument call to pyfreerdp when using clipboard

  • ocr filtering by Icon elements

3.1.4

Released on 20-05-2022

Fixed

  • error handling of internal errors when parsing pdf

3.1.3

Released on 18-05-2022

Fixed

  • bug when using cc field in send_mail

3.1.2

Released on 17-05-2022

Fixed

  • bug when regex match didn’t appear in unicode texts

  • robot now correctly releases shared memory

  • handling of email attachments with no content disposition

  • email sending to CC and BCC now works correctly

3.1.1

Released on 09-05-2022

Fixed

  • pdf text extraction now correctly uses clipping boxes

3.1.0

Released on 04-05-2022

Added

  • email-notification object for creating reports

  • ability to use non-default port when using RDPRobot

  • context-manager into clear_screen_masks command to work with with

  • sleep-time argument into rdp-robot start-process command

  • interval parameters to debugger

  • or_index variable into bound-box for result of Or search-object

  • BaseParser and ParserHandler as standardized way to implement document data extraction

  • image logs for key_* commands

Changed

  • debugger to use more processor cores to speedup logs generation

Fixed

  • rdp-robot start_process command issues

  • error when parsing log names

  • email sending from alias or shared mailbox

  • email query when used without parameters

  • email message operations when the message is not in the same path

  • too-many-elements exception in wait-for command

3.0.1

Released on 14-04-2022

Fixed

  • unknown method error in wait_for_clipboard_file_transfer command

3.0.0

Released on 09-04-2022

Added

  • missing RDPRobot clipboard functions.

  • several new SearchObject exceptions

  • ability to change name of the folder which is shared with rdp machine

  • text split by capital letters and special characters as “:”, “.”

  • PDFRobot which is used to read and parse PDF files.

Changed

  • calculation of area intersection between working-area and detected boxes

  • email connection handling

  • core structure for merging core & toolbox together

  • main logger from aivirocore into aiviro

  • location of toolbox modules into aiviro package

Fixed

  • regex correctly searching after normalization.

  • detection of a scene stability transition

  • element-index error in line-grouping service

  • documentation typos

  • documentation after aiviro + toolbox merge

  • waiting for rdp file transfer not working correctly

Removed

  • all deprecated funcionality of 2.x.x aiviro-core versions

  • unimportant warning messages from wait_* commands

2.6.4

Released on 23-03-2022

Added

  • command disable_cross_validation

  • gif generation from image buffer

  • ignore_timeout_exception argument for wait-until-disappear command

  • bitwise operators & (and), | (or) for search-objects

  • command close_tab for WebRobot

  • new TableAnchor options, that calculates distance between objects based on both axis

  • more advanced cache into recognition-service, reducing time to find elements

Changed

  • image hash in cache-service from percentual to sha1

  • cross-validation is skipped for Area & BoundBox elements

  • prefect version dependency to 1.0.0 from 0.15.6

Fixed

  • onTheRight/Left element search, reference object is removed from the find-results

  • debug-log for CustomBox search-object

  • handling of connection errors to vision server.

  • some typos in documentation

  • typo in text_to_type & check_are_in_row commands

  • cross-validation exception in wait-for command

  • error logs into prefect

  • merge of Input element with OCR text

  • behaviour of DISTANCE option in positional search-objects

2.6.3

Released on 14-12-2021

Added

  • option to run start_process as admin when using RDP robot

Fixed

  • issue when calculating disjunction of ocr and icon box, could lead to bounding box with negative width

2.6.2

Released on 08-11-2021

Added

  • print_port option into configuration file for video-stream

  • argument use_full_screen into working-area checkpoint commands

  • rolling logs that can be specified by maximum disk size, maximum number or both

  • set_working_area_by_grid command, which can be used for tables

Changed

  • error when image is not set in StaticRobot

Fixed

  • get-by-scroll command with timeout > 0

  • set-working-area-by-checkpoint when checkpoint is created with set working-area

  • incorrect mask generation of add_mask_by_display when all sides are set to True and include_object set to False

Removed

  • restriction of saving images locally from buffer in docker environment

  • unused tests & test-data from repository

2.6.1

Released on 11-10-2021

Added

  • aiviro-core version into robot’s printout info

  • limit-working-area command

  • exception section into documentation

  • extra requirement Prefect into setup.py

Changed

  • wait-* commands to work as get command when timeout is 0

  • command proxy now uses cached property

Fixed

  • get_by_scroll sometimes didn’t register scroll when application registered scroll slowly

  • required version of pyfreerdp package

  • changelog text & doc-strings

Removed

  • unused methods from robot

2.6.0

Released on 22-09-2021

Added

  • timezone option into config-file for aiviro-logs

  • create_static_robot which can be used when working with images without the need to create another type of robot

  • option to compress Aiviro-Logs

  • set_working_area_by_boundaries now supports multiple rules of bounding box creation

  • metadata-logging for Aiviro image-logs

  • option to generate only last-N-log-images

  • filtering of too big OCR boxes

  • get_by_scroll command which can be used to scroll until the all elements are found

  • support for new version of pyfreerdp (version 3.0.0)

  • new arguments for create_rdp_robot which allow shared folder

  • integration tests for RDP robot

  • documentation for aiviro-config file

  • automatic import sorting of repo

  • exception raise in set-working-area-by-checkpoint for images with different shape

  • automatic generation of CHANGELOG.md

  • collection of all release changes into one file CHANGELOG.md

Changed

  • filtering of OCR boxes using Icon elements

  • documentation examples to use interactive python correctly

  • versions of required dependency packages

  • start_process has now a parameter to check only in lower left corner of the screen for dialog windows. (Most likely place for the run window)

  • start_process waits for the dialog to disapear to continue execution

Fixed

  • text splitting for special characters in sentence

  • generation of documentation changelog

  • type_text argument call error

  • type-text command when input element is not found

Deprecated

  • parameters upper_boundary and lower_boundary of set_working_area_by_boundaries will are replaced with first_boundary and second_boundary

Removed

  • error logs in cross-validation for wait-until-disappear command

2.5.4

Released on 18-08-2021

Changed

  • doc-strings & documentation references

Fixed

  • working-area for get with static-image

  • log of after-image in type-text command

Removed

  • ReferenceObjectNotFound exception

2.5.3

Released on 28-07-2021

Added

  • AND search-object

Changed

  • version of pyfreerdp for extra-dependency

Fixed

  • too-nested-object exception

2.5.2

Released on 21-07-2021

Added

  • cwd argument for prefect-utils method run_shell_command

  • doc-string for EXCLUSIVE find-text-method option

  • unit-tests for devices/

  • static runner manager & get-elements command from image

Changed

  • aiviro setup options with extra python packages

  • reformat source-code with Black

Fixed

  • image resources for documentation

  • table-anchor, NO_THRESHOLD option for Above/Below positional elements

Removed

  • upload file-size restriction

2.5.1

Released on 15-07-2021

Fixed

  • regex normalization

  • missing keyboard for RDP-robot

2.5.0

Released on 12-07-2021

Added

  • filter of reference object in positional search-objects

  • markdown reporting into Prefect artifact

  • command to get streaming url

  • EXCLUSIVE text-find-method option

  • command to switch between robots as video-streaming source

  • RegexText search-object

  • option to create a robot from env-variable

  • support for CustomSearchObject & CustomBox

  • support for webdriver capability setting

Changed

  • constants value names to a shorter version (deprecated)

Fixed

  • text merging into lines

  • exact word search

  • reference-object & element-index exception for wait_* commands

  • checkpoint working-area command to work with current working-area

  • Above/Below anchor distance thresholds

Removed

  • deprecated functionality for 2.5.0 version

2.4.6

Released on 23-06-2021

Fixed

  • merging of different sized text

2.4.5

Released on 01-06-2021

Fixed

  • correct termination of Aiviro-script in Prefect

  • empty regex_match is not stored in BoundBox anymore

  • positional elements overlaping

  • processing of image-logs & add additional evaluation for wait_* command

2.4.4

Released on 19-05-2021

Changed

  • streaming settings

2.4.3

Released on 13-05-2021

Added

  • example code into documentation

Fixed

  • prefect logging

2.4.2

Released on 28-04-2021

Added

  • debug messages for cross-validation service

  • sensitive-data argument to clear-and-type command

  • SearchObjectError exception everywhere, when search-object is not found

  • prefect logging safeguard

  • non-case-sensitive option for aiviro.text_find_methods

Changed

  • updated prefect API for version 0.14.16

Fixed

  • clear-and-type command causing double-click

  • image-logging for INFO debug level & INFO level set as default

  • Icon object text-reference when texts are too close to each other

2.4.1

Released on 07-04-2021

Fixed

  • wait-for command for positional element

2.4.0

Released on 02-04-2021

Added

  • text content into output BoundBox

  • aiviro custom exceptions

  • argument for env variables for prefect-utils, run-aiviro-script method

  • move-away command, mouse moves over empty position at screen

  • artifacts integration with Prefect

  • asynchronous logging for stdout/stderr for Prefect Tasks

  • wait-until-disappear command

  • regex result into output BoundBox

Changed

  • optimize Wait-For command

Fixed

  • robot methods documentation

  • sub-image search service

  • configuration constants

  • cross-validation service

  • working-area checkpoint command for gray background

2.3.1

Released on 12-03-2021

Added

  • warning for image-buffer, transfer-file & forbid image-logging and streaming in docker

  • socket timeout for robot streaming

Changed

  • return state for NotifyTask

Fixed

  • context manager for working-area

2.3.0

Released on 09-03-2021

Added

  • image upload logs to dashboard

  • api-key check for Aiviro configuration

  • argument for addional files into image-upload logs

  • notification task for Prefect

  • tips & trciks section into Documentation

  • video streaming static link

Changed

  • stability checking limited by working-area

  • method of screen capture for cross-validation service

Fixed

  • small bugs & sleep-check

2.2.0

Released on 12-02-2021

Added

  • timeout for recieving a new image

  • image logging buffer

  • creation of working-area by checkpoint

Changed

  • logging of BoundBox object into human readable form

Fixed

  • sleep-check for type-text command

  • cache getting duplicated results